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ABSTRACT 


This  thesis  explores  the  top-level  requirements  for  an  Automated  Tool  for 
Acquisition  Program  Management  Students  (ATAPMS)  that  is  designed  to  enhance 
training  and  education  in  the  acquisition  management  field.  The  Department  of  Defense 
(DoD)  has  identified  the  education  and  training  of  the  acquisition  workforce  as  a  strategy 
to  help  make  the  acquisition  system  more  effective  and  efficient.  As  a  result,  the  DoD 
established  the  Defense  Acquisition  University  (DAU)  to  provide  the  required  education 
and  training.  More  recently,  EO  13111  and  the  Defense  Reform  Initiative  have  presented 
a  mandate  for  the  DoD  to  find  ways  to  use  technology  to  further  this  strategy. 

Currently,  the  consortium  schools  of  the  DAU  are  using  emerging  technologies  to 
increase  access  to  their  courses.  However,  the  DAU  curricula  lack  automated  acquisition 
management  training  programs  that  allow  instructors  to  qualitatively  assess  students' 
work. 

This  thesis  recommends  a  set  of  top-level  requirements  for  an  automated  program 
that  are  in  compliance  with  the  Advanced  Distance  Learning  Initiative.  It  then  illustrates 
through  a  prototype  module,  using  a  commercial  authoring  tool,  how  an  ATAPMS  can 
assist  the  DAU  instructors  teach  the  critical  aspects  of  Acquisition  Program  Management. 
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I. 


INTRODUCTION 


A.  AREA  OF  RESEARCH 

The  primary  focus  of  this  research  is  to  explore  the  top-level  design  requirements 
for  an  automated  education  and  training  program  for  acquisition  management  topics.  The 
purpose  of  this  automated  program  is  to  enhance  the  training  and  education  in  the 
acquisition  management  field.  This  thesis  recommends  a  set  of  top-level  requirements 
and  then  illustrates,  by  developing  a  prototype,  how  an  automated  program  can  assist  the 
Defense  Acquisition  University  (DAU)  consortium  instructors  teach  the  critical  aspects  of 
Acquisition  Program  Management. 

B.  BACKGROUND 

The  Department  of  Defense  includes  improvement  to  education  and  training  as 
one  of  the  goals  of  Acquisition  Reform  [Ref.  1].  Additionally,  Executive  Order  13111 
[Ref.  2]  directs  federal  agencies  to  take  certain  steps  to  enhance  an  employee's  training 
opportunities  through  the  use  of  technology.  The  DAU,  through  its  14  consortium  schools 
which  include  the  Naval  Postgraduate  School  (NPS),  provides  education  and  training  for 
acquisition  professionals  throughout  the  Department  of  Defense.  Currently,  the  DAU  is 
using  emerging  technologies  to  increase  access  to  its  courses.  However,  the  DAU 
curricula  lack  acquisition  management  training  programs  that  are  dynamic  or  that  allow 
instructors  to  qualitatively  assess  students'  work. 

C.  RESEARCH  QUESTIONS 

The  primary  research  question  addressed  in  this  thesis  is: 
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-What  are  the  top-level  requirements  for  developing  an  automated  program  that 
assists  the  DAU  instructors  with  teaching  the  critical  aspects  of  Acquisition  Program 
Management? 

Subsidiary  research  questions  are: 

-What  are  the  existing  automated  training  programs  for  Acquisition  Program 
Management  students? 

-What  are  the  critical  aspects  of  Acquisition  Program  Management? 

-To  what  extent  are  they  adequately  presented  in  the  existing  automated  training 
tools? 

-What  are  the  needs  of  the  DAU  consortium  instructors  that  can  be  incorporated 
in  an  automated  program? 

-How  may  these  design  requirements  be  implemented  in  a  prototype? 

D.  METHODS 

The  first  objective  of  this  research  is  to  document  the  critical  aspects  of 
Acquisition  Program  Management  and  provide  an  overview  of  the  existing  automated 
training  tools  used  to  teach  them.  This  is  accomplished  through  personal  interviews  and  a 
literature  review  of  sources  including: 

-Staff  members  and  instructors  at  several  of  the  Defense  Acquisition  University 
consortium  schools,  including  the  Naval  Postgraduate  School. 

-References  and  publications  available  at  the  Naval  Postgraduate  School  library. 
-Internet  websites  (DoD,  academic,  and  commercial). 

The  second  objective  is  determining  the  top-level  requirements  for  developing  an 
automated  program  that  assists  the  DAU  consortium  instructors  with  teaching  the  critical 
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aspects  of  Acquisition  Program  Management.  This  is  also  accomplished  through  personal 
interviews  and  a  literature  review  of  sources  including: 

-Instructors  at  the  Naval  Postgraduate  School. 

-Internet  websites  (DoD,  academic,  and  commercial). 

After  developing  these  top-level  requirements,  this  thesis  illustrates  them  using  a 
prototype  of  a  course  module. 

E.  ORGANIZATION 

Chapter  II  (Background)  explores  the  DoD's  efforts  to  improve  the  acquisition 
system  though  the  proper  training  and  education  of  the  acquisition  workforce.  It  also 
details  the  latest  developments  in  expanding  the  reach  and  reducing  the  cost  of  providing 
education  and  training  through  the  development  of  standards  and  the  use  of  technology. 
Finally,  the  chapter  identifies  the  need  for  an  application  that  possess  a  feedback 
mechanism  that  enables  an  instructor  to  assess  a  user's  opinion  or  overall  understanding 
of  concepts. 

Chapter  m  (Program  Management)  provides  an  overview  of  program 
management  using  an  industry  model  to  present  the  related  functional  disciplines  that  a 
program  manager  is  expected  to  integrate  during  the  program  life  cycle.  It  also  introduces 
the  concept  of  an  "Automated  Tool  for  Acquisition  Program  Management  Students" 
(ATAPMS)  to  assist  the  DAU  instructors. 

Chapter  IV  (Structure  and  Content  Requirements)  develops  the  structure, 
strategies  and  topics  for  an  ATAPMS  tailored  to  the  Naval  Postgraduate  School 
instructors'  needs. 
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Chapter  V  (Design  of  the  Program  Management  Training  Tool)  presents  one 
possible  approach  to  devising  an  ATAPMS  application  that  meets  the  requirements 
specified  in  Chapter  Four.  The  chapter  enumerates  strategies  for  acquiring  the  software 
required  to  support  an  ATAPMS  application  and  then  uses  one  of  those  strategies  for 
demonstrating  the  construction  of  a  notional  module. 

Chapter  VI  (Conclusions  and  Recommendations)  summarizes  the  findings  of  the 
research  and  presents  recommendations  for  further  research  and  study. 

F.  BENEFITS  OF  STUDY 

This  study  provides  the  requirements  and  a  design  for  an  automated  tool  that  once 
implemented,  can  be  used  to  enhance  the  Naval  Postgraduate  School's  capability  to 
provide  instruction  on  Acquisition  Program  Management.  This  application  is  also 
relevant  to  the  Defense  Acquisition  University  consortium  schools  because  it  allows  the 
host  institution  to  offer  a  portable,  comprehensive,  and  thoroughly  relevant  automated 
program  through  which  students  can  demonstrate  their  knowledge  on  the  fundamental 
aspects  of  Acquisition  Program  Management.  The  program  also  adheres  to  the  precepts  of 
the  Defense  Reform  Initiative,  the  Advanced  Distance  Learning  Initiative,  and  Executive 
Order  13111,  "Using  Technology  to  Improve  Training  Opportunities." 


II.  BACKGROUND 


A.  INTRODUCTION 

This  chapter  illustrates  the  need  to  execute  the  Defense  Acquisition  mission  as 
effectively  and  efficiently  as  possible.  It  also  explores  the  Department  of  Defense's  (DoD) 
efforts  to  improve  the  acquisition  system  though  the  proper  training  and  education  of  the 
acquisition  workforce.  Finally,  this  chapter  details  the  latest  developments  in  expanding 
the  reach  and  reducing  the  cost  of  providing  education  and  training  through  the 
development  of  standards  and  the  use  of  technology. 

B.  MOTIVATION  FOR  BETTER  MANAGEMENT 

The  Defense  Acquisition  mission  is  extensive  and  complex  and  requires  a 

significant  amount  of  the  DoD's  resources.  The  scope  of  this  acquisition  mission  covers: 

The  conceptualization,  initiation,  design,  development,  test,  contracting, 
production,  fielding,  deployment,  and  logistic  support,  modification,  and 
disposal  of  weapon  and  other  systems,  supplies,  or  services  (including 
construction)  to  satisfy  DoD  needs,  intended  for  use  in  or  in  support  of 
military  missions.  [Ref.  3] 

Every  year  the  DoD  spends  approximately  $80  billion  in  Research,  Development,  Test 
and  Evaluation  (RDT&E)  and  Procurement  appropriated  funds  for  these  acquisition- 
related  expenses,  or  approximately  30%  of  the  overall  DoD  annual  budget  [Ref.  4:  p.  6]. 

Over  the  last  15  years,  the  DoD  has  experienced  a  trend  of  shrinking  budgets 
(Figure  2-l)[Ref.  4:  p.  89].  While  fiscal  year  2000  to  fiscal  year  2004  axe  expected  to 
provide  a  slight  reversal  of  the  downward  trend,  the  DoD  budget  as  an  overall  percentage 
of  the  Federal  budget  remains  historically  low  (Figure  2-2)  [Ref.  5].  With  such  a 
significant  percentage  of  DoD  capital  expended  in  the  area  of  acquisition,  especially  in 
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view  of  the  current  budgetary  trend,  there  is  a  need  to  be  as  effective  and  efficient  as 


possible  when  acquiring  the  required  goods  and  services. 
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Figure  2-1.  DoD  Total  Obligation  Authority. 


Figure  2-2.  Defense  Outlay  as  a  Percentage  of  the  Overall  Federal  Budget. 


C.  DAWI A  AND  DAU 

One  method  the  DoD  uses  to  improve  the  overall  acquisition  system  is  to  improve 
the  training  and  education  of  the  people  who  work  in  the  acquisition  field.  The  standards 
and  infrastructure  used  for  the  training  and  education  provided  today  were  established  in 


1990  through  the  Defense  Acquisition  Workforce  Improvement  Act  (DAWLA),  Public 
Law  101-501,  Title  XII.  The  law  directed  the  following  [Ref.  6:  p.  1638]: 

-Establishment  of  policies  and  procedures  for  acquisition  personnel 
education  and  training  programs,  including  intern,  cooperative  education, 
and  scholarship  programs. 

-Authorization  of  special  pay  for  acquisition  officers  in  certain  critical 
positions,  and  repayment  of  student  loans  to  facilitate  recruitment  and 
retention  of  acquisition  personnel. 

-Revision  of  policies  and  procedures  for  the  recruitment,  training,  and 
career  development  of  military  and  civilian  defense  acquisition  personnel. 

-Establishment  of  acquisition  corps  for  each  military  department  and  for 
other  defense  agencies. 

-Establishment  of  the  Defense  Acquisition  University. 

The  DoD  is  implementing  the  law  through  the  adoption  of  standards  detailed  in 
DoD  manual  5000.52-M  "Acquisition  Career  Development  Program"  released  in 
November  22,  1995.  The  manual  "provides  information,  guidance,  and  requirements  for 
improving  the  management  and  professionalism  of  the  acquisition  workforce"  [Ref.  7:  p. 
14]. 


As  directed  by  the  DAWIA,  the  DoD  created  the  Defense  Acquisition  University 
(DAU)  in  1990.  The  DAU  is  responsible  for  coordinating  the  provision  of  the  mandatory, 
assignment  specific,  and  continuing  education  courses  that  meet  the  standards  prescribed 
by  DoD  5000.52-M  for  military  and  civilian  personnel  serving  in  several  acquisition 
career  fields.  The  DAU's  mission  is  to  educate  and  train  professionals  for  effective 
service  in  the  Defense  acquisition  system.  [Ref.  8] 

The  DAU  was  established  as  a  consortium  of  12  DoD  educational  and  training 
institutions  located  throughout  the  country.  These  members  provide  more  than  85 
acquisition  courses  to  the  approximate  106,000  entry,  intermediate,  and  senior  level 
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civilian  and  uniformed  personnel  [Ref.  9:  p.  3]  to  allow  them  to  attain  certification  in  one 

or  more  of  the  following  1 1  defense  acquisition  career  fields  [Ref.  9:  p.  27]: 

-Acquisition  Logistics 
-Auditing 

-Business,  Cost  Estimating,  and  Financial  Management 

-Communications-Computer  Systems 

-Contracting 

-Industrial  and  /or  Contract  Property  Management 
-Manufacturing  and  Production 
-Program  Management 
-Purchasing 

-Systems  Planning,  Research,  Development  and  Engineering 
-Test  and  Evaluation 

As  of  1999,  the  consortium  members  providing  the  training  and  education  to  the 
acquisition  workforce  were  [Ref.  9:  p.  4]: 

-Air  Force  Institute  of  Technology  (AFIT) 

-Lackland  Training  Facility 

-Army  Logistics  Management  College  (ALMC) 

-Defense  Logistics  Agency  Center  for  Training,  Education  and  Development 
(formerly  DCPSO) 

-Defense  Contract  Audit  Institute  (DCAI) 

-Defense  Systems  Management  College  (DSMC) 

-Information  Resource  Management  College  (IRMC) 

-Industrial  College  of  the  Armed  Forces  (ICAF) 

-Naval  Center  for  Acquisition  Training  (NCAT) 

-Naval  Postgraduate  School  (NPS) 

-Naval  Facilities  Contracting  Training  Center  (NFCTC) 

-Naval  Warfare  Assessment  Division  (NWAD) 

D.  MANDATES  FOR  TECHNOLOGY 

Although  today's  acquisition  training  and  education  infrastructure  was  originally 
shaped  by  the  passage  of  DAWIA,  that  infrastructure  is  continuously  influenced  by 
developments  affecting  education  and  training  within  the  Federal  government.  One  of 
these  developments  is  the  drive  to  leverage  technology  as  a  way  to  reduce  cost  and 
increase  the  availability  of  training  and  education  opportunities. 
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1.  Defense  Reform  Initiative 

The  Secretary  of  Defense  introduced  the  Defense  Reform  Initiative  (DRI)  on 
November  10th,  1997  to  improve  business  practices  in  the  Department  of  Defense.  The 
DRI  is  described  by  the  DoD  as  an  effort  to  coordinate  several  internal  and  external 


ongoing  reform  initiatives  and  to  bring  the  Department  "in  tune  with  the  times”.  The 
Secretary’s  intent  was  to  use  the  DRI  as  a  mechanism  to  identify  savings  and  migrate 
resources  to  support  modernization  accounts.  [Ref.  1] 
a)  DRI  Goals 

The  DRI  originally  focused  on  initiatives  in  four  main  areas,  and 

eventually  expanded  to  include  nine  additional  elements.  The  initial  areas  were  [Ref.  1]: 

-Reengineer:  Adopt  modem  business  practices  to  achieve  world-class 
standards  of  performance; 

-Consolidate:  Streamline  organizations  to  remove  redundancy  and 
maximize  synergy; 

-Compete:  Apply  market  mechanisms  to  improve  quality,  reduce  costs, 
and  respond  to  customer  needs; 

-Streamline:  Reduce  excess  support  structures  to  free  resources  and  focus 
on  core  competencies. 

The  following  nine  elements  are  the  expanded  areas,  one  of  which  includes 
Acquisition  [Ref.  1]: 

-Adopting  Best  Business  Practices 

-Quality  of  Life 

-Organizational  Streamlining 

-Competition 

-Infrastructure 

-Logistics 

-Cyberspace 

-Homeland  Defense 

-Acquisition 
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b)  Acquisition  Element 

Acquisition  Reform  supports  the  DoD's  efforts  to  provide  the  warfighter 
with  products  that  "work  better,  cost  less,  and  are  obtained  from  a  globally  competitive 
national  industry  base"  [Ref.  1].  The  major  goals  of  the  acquisition  reform  effort  include 
[Ref.  1]: 

-Fostering  Partnership:  Modernizing  defense;  partnering  with 
communities;  decreasing  paper  transactions;  and  reducing  toxic  pollution. 

-Internal  Reinvention:  Streamlining  our  workforce;  providing  effective 
cost  accounting;  reducing  excess  inventory;  and  minimizing  weapons  cost 
growth. 

-Delivering  Great  Service:  New  weapons  in  less  time;  better  logistic 
supply  services;  simplifying  buying  of  goods  and  services;  and  educating 
and  training  the  defense  acquisition  workforce. 

The  principal  controlling  documents  for  Acquisition  programs  are  DoD 
Directive  5000.1;  "Defense  Acquisition",  and  DoD  Regulation  5000.2-R;  "Mandatory 
Procedures  for  Major  Defense  Acquisition  Programs  (MDAPs)  and  Major  Automated 
Information  Systems  (MAISs)  Acquisition  Programs."  DoDD  5000.1  describes  broad 
management  principles  that  are  applicable  to  all  DoD  acquisition  programs.  DoD  5000.2- 
R  establishes  mandatory  procedures  for  MDAPs  and  MAISs.  [Ref.  10] 

As  part  of  the  acquisition  reform  effort,  the  DoD  issued  an  update  to  the  5000 
series  Directive  and  Regulation.  This  update  defines  an  acquisition  environment  that 
makes  DoD  "the  smartest,  most  responsive  buyer  of  the  best  goods  and  services  that  meet 
our  warfighters'  needs,  at  the  best  dollar  value  over  the  life  of  the  product  to  help  meet 
these  efforts"  [Ref.  10].  The  updated  documents  contained  six  major  themes  that  support 
the  acquisition  reform  effort  [Ref.  10]: 
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-Teamwork:  We  must  work  together  as  a  team  to  build  successful 
programs,  identify  problems  early,  and  maintain  a  cooperative  spirit  of 
resolution,  thereby  providing  programs  the  highest  opportunity  for  success. 
-Tailoring:  Milestone  Decision  Authorities  (MDA  should  strive  to  tailor 
most  aspects  of  the  acquisition  process,  including  program  documentation, 
acquisition  phases,  and  the  timing,  scope,  and  level  of  decision  reviews. 
-Empowerment:  Program  managers  do  not  have  to  ask  permission  to  take 
actions  that  are  otherwise  permitted  by  law  and  are  within  the  scope  of 
their  charters.  The  Department  has  long  relied  on  volumes  of  guidance  and 
regulation,  prescribing  every  detail  of  both  process  and  documentation. 

DoD  has  also  had  a  habit  of  dealing  with  industry  through  a  rigid  system 
of  military  specifications.  DoDD  5000.1  and  DoD  5000.2-R  reflect  current 
efforts  to  empower  our  people  and  our  vendors  to  do  the  best  they  can. 

-Cost  as  an  Independent  Variable  (CAIV):  The  acquisition  process 
described  in  DoDD  5000.1  and  DoD  5000.2-R  must  consider  both 
performance  requirements  and  fiscal  constraints.  Accordingly,  cost  must 
also  be  an  independent  variable  in  programmatic  decisions,  with 
responsible  cost  objectives  set  for  each  program  phase. 

-Commercial  Products:  Historically,  DoD  has  relied  on  segments  of  the 
U.S.  technology  and  industrial  base  principally  dedicated  to  supporting 
DoD  requirements... Acquisition  of  commercial  items,  components, 
processes,  and  practices  provides  rapid  and  affordable  application  of  these 
technologies  to  validated,  DoD  mission  needs. 

-Best  Practices:  DoD  5000.2-R  describes  a  simplified  and  flexible 
management  process,  modeled  on  sound  business  practices.  Acquisitions 
of  the  future  must  take  into  account  customary  commercial  practices  in 
developing  acquisition  strategies  and  contracting  arrangements. 

"Educating  and  training  the  workforce"  is  a  subset  to  one  of  the  major  goals  of  the 

Acquisition  element.  In  an  effort  to  support  the  acquisition  reform  education  and  training 

goal,  the  DAU  is  planning  to  use  emerging  technologies  to  increase  access  to  its  courses. 

Further,  in  its  "Implementation  Plan  for  Technology  Based  Education  and  Training" 

document,  the  DAU  recognizes  that  these  courses  need  to  be  "designed  and  delivered  so 

they  are  updated  quickly  and  inexpensively  and  their  shelf  life  can  be  extended."  [Ref.  11] 

Currently,  the  DAU  is  working  toward  these  efforts  and  taking  advantage  of  new 

distributed  learning  technology  to  modernize  its  curriculum.  In  1997,  10%  of  the  DAU's 

courses  were  modernized  and  delivered  by  distributed  learning  technologies.  The  goal  is 
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to  complete  modernization  of  all  of  its  courses  through  technology  enhancements  by 
2003.  [Ref.  12] 

2.  Executive  Order  13111 

President  Clinton  signed  Executive  Order  (EO)  13111,  "Using  Technology  to 
Improve  Training  Opportunities",  on  12  January  1999.  This  EO  directs  federal  agencies  to 
take  certain  steps  to  enhance  an  employee's  training  opportunities  through  the  use  of 
technology.  Specifically,  the  Order  states  "A  coordinated  federal  effort  is  needed  to 
provide  flexible  training  opportunities  to  employees  and  to  explore  how  federal  training 
programs,  initiatives,  and  policies  can  better  support  lifelong  learning  through  the  use  of 
learning  technology."  To  assist  with  this  goal,  the  EO  established  the  President's  Task 
Force  on  federal  training  technology  and  created  an  advisory  committee  on  expanding 
training  opportunities.  [Ref.  2] 

3.  Advanced  Distance  Learning  Initiative 

The  DoD  launched  the  Advanced  Distance  Learning  (ADL)  Initiative  in 
November  of  1997.  The  ADL  is  an  effort  by  the  DoD  to  identify  more  efficient  and 
effective  ways  to  educate  and  train  the  DoD  personnel  through  the  development  of 
advanced  distributed  learning  technologies.  The  purpose  of  the  ADL  initiative  is  to 
"ensure  access  to  high-quality  education  and  training  materials  that  can  be  tailored  to 
individual  learner  needs  and  can  be  made  available  whenever  and  wherever  they  are 
required."  [Ref.  13] 

In  his  November  1998  memorandum  "Developing  and  Implementing  the  DoD 
Advanced  Distributed  Learning  (ADL)  Initiative",  the  Deputy  Secretary  of  Defense 
directed  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD(P&R))  to 
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lead  the  ADL  Initiative  and  to  work  with  the  Services,  Joint  Staff,  and  other  DoD 
components  in  developing  and  implementing  ADL  technologies  across  the  DoD  [Ref. 
14].  Through  this  initiative,  the  DoD  seeks  to  accelerate  large-scale  development  of 
"dynamic  and  cost  effective  learning  software.. .in  order  to  meet  the  education  and 
training  needs  of  the  military  and  the  nation's  workforce  in  the  21st  century."  [Ref.  13] 

a)  Lack  of  Standards 

This  initiative  addresses  the  problem  created  by  a  lack  of  standards  within 
the  education  software  industry.  Developers  of  online  learning  resources  have  many 
different  software  tools  to  choose  from  to  create  their  courses.  Prior  to  the  ADL,  however, 
there  was  no  common  learning  management  software  reference  model.  As  a  result,  the 
available  computer-based  training  authoring  tools  have  historically  been  developed  on  a 
proprietary,  company  by  company  basis.  This  scenario  results  in  high  development  costs 
and  limits  the  reuse  potential  for  already  developed  software.  [Ref.  15] 

b)  Instructional  Management  Systems  Project 

To  help  alleviate  the  commonality  problem  within  the  DoD  and 
accomplish  the  previously  stated  DoD  objectives,  the  DoD  has  teamed  with  the 
Instructional  Management  System  (IMS)  Project.  The  IMS  is  a  consortium  of  government 
organizations,  over  1,600  colleges  and  universities,  and  150  corporations  [Ref.  13].  The 
IMS  project  is  an  effort  to  develop  next-generation  open  architecture  for  online  learning 
software. 

Overall,  the  IMS  attempts  to  address  three  obstacles  for  providing 
effective  online  materials  and  learning  environments  [Ref.  16]: 
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-Lack  of  standards  for  locating  and  operating  interactive  platform- 
independent  materials 

-Lack  of  support  for  the  collaborative  and  dynamic  nature  of  learning 
-Lack  of  incentives  and  structure  for  developing  and  sharing  content 

By  teaming  with  the  IMS  consortium,  the  DoD  is  helping  to  formulate  industry  - 

wide  voluntary  learning  specification  guidelines  that  will  make  learning  software 

interoperable,  reusable,  and  accessible  through  the  Internet  [Ref.  15].  The  IMS  technical 

specifications  will  provide  the  general  guidelines  and  requirements  developers  must  write 

to  for  creating  interoperable  content  and  management  systems  [Ref.  17]. 

The  IMS  efforts  are  centered  on  specifications  in  four  areas  [Ref.  17]: 

-Meta-data:  Descriptive  labels  can  be  used  to  index  learning  resources  to 
make  them  easier  to  find  and  use.  Such  labels  are  "data  about  data"  and 
are  referred  to  as  "meta-data."  An  example  of  meta-data  is  the  label  on  a 
can  of  soup,  which  describes  the  can’s  ingredients,  weight,  cost,  and  so 
forth...A  meta-data  specification  makes  the  process  of  finding  and  using  a 
resource  more  efficient  by  providing  a  structure  of  defined  elements  that 
describe,  or  catalog,  the  learning  resource,  along  with  requirements  about 
how  the  elements  are  to  be  used  and  represented.  [Ref.  18] 

-Packaging  and  Run-time  services:  This  area  focuses  on  specifications  for 
searching,  digital  packaging,  and  modularizing  content  with  the  goal  that 
any  content  can  run  on  any  learning  server.  Resources  can  be  drawn  from 
any  developer  and  potentially  reused  in  different  learning  activities.  [Ref. 

19] 

-Profiles:  User  profiles  are  mobile,  user-controlled  collections  of  personal 
and  educational  data  including  personal  information,  performance 
information,  and  preference  information.  This  data  represents  a  rich 
resource  that  a  user  can  draw  on  to  facilitate,  customize,  and  manage  his  or 
her  learning  experience(s).  [Ref.  19] 

-Enterprise  integration:  The  IMS  Enterprise  Information  Model  describes 
data  structures  that  are  used  to  provide  interoperability  of  Internet-based 
Instructional  Management  systems  with  other  Enterprise  systems  used  to 
support  the  operations  of  an  organization.  The  objective  of  the  IMS 
Enterprise  Information  Model  is  to  define  a  standardized  set  of  structures 
that  can  be  used  to  exchange  data  between  different  systems.  These 
structures  provide  the  basis  for  standardized  data  bindings  that  allow 
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software  developers  and  implementers  to  create  Instructional  Management 

processes  that  interoperate  across  systems  developed  independently  by 

various  software  developers.  [Ref.  20] 

E.  CURRENT  METHODS  USED  TO  ASSIST  ACQUISITION  TRAINING 

The  DAU  is  using  technology-based  education  and  training  methods  to  maintain 
pace  with  the  reforms  in  the  acquisition  process.  While  the  DAU  has  successfully 
converted  several  of  its  courses  to  a  technology-based  delivery,  the  methods  for 
presenting  the  acquisition  related  training  and  education  are  still  as  diverse  as  the 
consortium  members  providing  it  [Ref.  9:  p  i].  A  survey  by  the  author  of  the  delivery 
medium  used  by  the  consortium  members  revealed  several  methods  of  course 
presentation  including  traditional  classroom,  mobile  teaching,  video  teleconferencing, 
automated  programs,  and  the  Internet. 

1.  Classroom 

Classroom  training  is  the  traditional  method  of  providing  instruction.  Students 
typically  travel  to  the  institution's  location  from  several  dispersed  areas  to  receive  training 
and  instruction. 

2.  Mobile  Teaching 

One  of  the  non-traditional  methods  of  providing  instruction  to  the  students  is 
mobile  teaching.  Using  the  mobile  teaching  concept,  the  instructor  travels  to  the  student's 
location  instead  of  the  students  traveling  to  the  instructor's  location. 

3.  Video  Teleconference  Courses 

Several  consortium  members  conduct  courses  using  video  teleconference 
technology  as  the  medium.  The  typical  scenario  consists  of  an  instructor  co-located  at  one 
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location  with  a  small  contingency  of  students,  while  students  communicate  with  the 
instructor  via  real-time  voice  and  video  from  remote  locations. 

4.  Non-interactive  Automated  Courses 

Some  of  the  consortium  members  offer  course  material  in  a  non-interactive 
automated  format.  One  member  provides  a  laptop  computer  that  has  course  material 
preloaded.  The  course  material  is  presented  in  a  dynamic,  hypertext-linked  environment. 
This  program  is  not  interactive,  however,  as  its  purpose  is  to  present  information  instead 
of  soliciting  information  from  the  user. 

The  NPS  uses  a  program  called  Systems  Acquisition  For  Executives  (SAFE)  that 
works  on  a  slightly  different  premise.  Instead  of  automated  course  material,  SAFE  users 
only  have  access  to  a  workbook.  The  instructor  maintains  access  to  the  actual  SAFE 
program  and  provides  the  students  with  the  results  of  the  SAFE  program  simulations. 

SAFE  is  a  computer-assisted  exercise  that  allows  the  user  to  influence  the 
outcome  of  the  acquisition  program  in  the  scenario  based  on  decisions  concerning  the 
cost,  schedule,  and  performance  parameters.  The  scenario  and  worksheets  are  contained 
in  a  manual  and  the  students  use  this  information  to  provide  the  instructor  with  their  cost, 
schedule,  and  performance  decisions  at  discrete  points  through  the  exercise.  When  the 
instructor  enters  the  students'  data  into  the  automated  program,  the  SAFE  program 
simulates  an  outcome  and  then  generates  a  response  indicating  the  updated  status  of  each 
student's  program.  The  students  begin  another  session  of  decision-making  based  on  the 
updated  status  of  their  program  and  the  next  set  of  alternatives  presented  by  the  scenario 
in  the  workbook. 
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5.  Interactive  Internet  Courses 


The  DAU  provides  several  courses  using  the  Internet  or  a  compact  disk  as  the 
medium.  Each  course  contains  several  lessons  that  are  further  organized  by  topics.  These 
courses  are  self-paced,  interactive  modules.  They  contain  hypertext-links  that  allow 
students  to  navigate  through  instructional  material  and  case  scenarios.  The  interactive 
multi-media  capable  modules  require  students  to  input  information  gleaned  from  the 
instructional  text  and  case  scenarios.  The  modules  provide  the  user  feedback  based  on  the 
information  entered.  All  of  this  information  is  viewed  on  the  user's  web  browser.  These 
courses  typically  have  associated  tests  delivered  in  electronic  format  that  students  are 
required  to  pass  before  earning  credit  for  completing  the  course. 

F.  CURRENT  DEFICIENCIES 

The  intent  of  the  Executive  Order,  the  DRI,  and  the  ADL  initiative  is  clear:  the 
DAU  must  leverage  the  advances  of  technology  and  incorporate  automation  and  distance 
learning  into  the  training  and  education  of  the  acquisition  workforce  to  highest  degree 
possible. 

Even  though  the  DAU  has  a  plan  for  achieving  this  objective  and  the  number  of 
interactive  automated  courses  continue  to  grow,  the  current  courses  lack  the  ability  to 
provide  education  and  training  on  a  qualitative  basis.  The  responses  from  the  interactive 
courses  are  predetermined  and  codified  in  the  software.  There  is  no  interaction  with  a 
"subject  matter  expert"  to  review  the  student's  input  and  provide  a  qualitative  response  for 
scenarios  that  have  more  than  one  correct  answer.  In  essence,  while  courses  do  interact 
with  the  user,  they  do  not  possess  a  feedback  mechanism  to  assess  a  user's  opinion  or 
overall  understanding  of  concepts. 
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G.  CHAPTER  SUMMARY 

Because  the  acquisition  mission  is  broad  and  consumes  a  significant  percentage  of 
the  DoD's  resources,  there  is  an  unquestionable  need  to  execute  that  mission  in  the  most 
effective  and  efficient  manner  as  possible.  Accordingly,  the  DoD  has  identified  the  proper 
education  and  training  of  the  acquisition  workforce  as  a  strategy  to  help  improve  the 
acquisition  system.  As  directed  by  the  DAWIA,  the  DOD  established  the  DAU  to  provide 
this  required  education  and  training. 

There  is  a  current  emphasis  on  leveraging  the  use  of  technology  to  broaden  the 
accessibility  and  reduce  the  cost  of  providing  the  required  education  and  training  to  the 
acquisition  workforce.  EO  13111  and  the  DRI  provide  a  clear  mandate  for  the  DoD  to 
find  ways  to  use  technology  to  further  this  goal.  Additionally,  the  efforts  that  the  DoD 
have  undertaken  in  the  ADL  initiative  will  help  provide  the  mechanism  for  the  added 
reach  and  the  decreased  cost  through  the  creation  of  industry-wide  standards  for 
educational  software. 

Even  though  the  DAU  is  responding  to  this  mandate  and  providing  interactive 
distance  learning  courses,  there  is  one  type  of  course  not  yet  offered.  There  are  currently 
no  automated  courses  that  allow  for  qualitative  assessments  of  a  student's  work. 

The  next  chapter  provides  an  overview  of  one  of  the  critical  acquisition  career 
fields:  program  management.  Because  program  management  is  a  diverse  field  that 
integrates  the  other  disciplines  within  acquisition,  this  thesis  will  use  it  as  the  subject  of 
an  example  course  in  a  subsequent  chapter. 


ffl.  PROGRAM  MANAGEMENT 


A.  INTRODUCTION 

This  chapter  provides  an  overview  of  program  management.  The  chapter  uses  a 
model  borrowed  from  industry  to  present  the  related  functional  disciplines  that  a  program 
manager  is  expected  to  integrate  during  the  program  life  cycle.  The  disciplines  presented 
in  this  chapter  will  be  linked  to  their  Defense  Acquisition  counterparts  in  the  following 
chapter. 

B.  IMPORTANCE  OF  PROGRAM  MANAGEMENT 

Program  management  is  listed  as  one  of  the  1 1  critical  career  fields  within  the 
acquisition  system.  Specifically,  the  DoD  defines  program  management  as  [Ref.  21:  p. 
12]: 

The  process  whereby  a  single  leader  exercises  centralized  authority 
and  responsibility  for  planning,  organizing,  staffing,  controlling,  and 
leading  the  combined  efforts  of  participating/assigned  civilian  and  military 
personnel  and  organizations,  for  the  management  of  a  specific  defense 
acquisition  program  or  programs,  through  development,  production, 
deployment,  operations,  support,  and  disposal. 

Program  management  is  critical  to  the  acquisition  system  because  it  provides  a 
single  point  of  contact  who  acts  as  an  "integrator"  of  a  complex  system  of  "differing  but 
related  functional  disciplines"  [Ref.  21:  p.  13].  These  functional  disciplines  relate  to  areas 
such  as  business  and  financial  management,  logistics,  systems  engineering,  software 
management,  test  and  evaluation,  manufacturing  management  and  others  [Ref.  21:  p.  13]. 
To  provide  a  more  detailed  understanding  of  program  management,  the  remainder  of  this 
chapter  is  dedicated  to  presenting  an  overview  of  program  management  and  its  related 
functional  disciplines. 
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C.  PROGRAM  MANAGEMENT  OVERVIEW 

Industry  typically  uses  a  program  management  model  similar  to  that  used  in  the 
DoD  acquisition  system.  The  contractor  staff  usually  operates  its  program  office  to 
parallel  the  military  office  it  is  supporting  on  a  defense  project  [Ref.  21 :  p.  14]. 

Because  the  industry  and  defense  models  of  program  management  are  similar,  this 
chapter  uses  an  industry  model  from  Forsberg,  Mooz,  and  Cotterman's  textbook 
"Visualizing  Project  Management"  as  a  reference  to  describe  the  "differing  but  related 
functional  disciplines"  that  a  program  manager  must  integrate. 

While  the  program  management  models  are  similar,  the  terminology  is  not  always 
the  same.  The  DoD  uses  the  term  "program  manager"  and  industry  commonly  uses  the 
term  "project  manager".  Even  though  the  two  terms  are  synonymous  [Ref.  21:  p.  12],  the 
remainder  of  this  section  uses  the  term  "project  manager"  because  that  is  the  term  used  by 
the  authors. 

In  their  textbook,  Forsberg,  Mooz,  and  Cotterman  divide  the  project  management 
process  into  four  essentials:  Common  Vocabulary,  Teamwork,  Project  Cycle,  and  the 
Project  Management  Elements  [Ref.  22:  p.  27],  The  first  two  components,  Common 
Vocabulary  and  Teamwork,  are  characterized  as  perpetual  properties.  They  are  always 
present  throughout  the  lifecycle  of  a  project.  The  third  component.  Project  Cycle,  relates 
to  sequentially  occurring  activities  of  a  project.  Finally,  the  Project  Management 
Elements  are  described  as  situational  and  are  applied  on  an  as  needed  basis. 

1.  Common  Vocabulary 

A  common  vocabulary  is  the  first  perpetual  property  of  projects.  The  authors 
contend  that  to  succeed  at  project  management,  one  must  communicate  using  clearly 
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defined  terms.  Otherwise,  vocabulary  problems  can  lead  to  conflict  and  ultimately  destroy 
teamwork.  Therefore,  a  common  vocabulary  is  necessary  before  you  can  develop 
teamwork. 

This  is  true  in  part  because  problems  occur  when  terms  used  in  project 
management  are  confused  with  similar  terms  used  in  the  context  of  other  fields.  An 
example  given  is  the  contractor  who  mistook  the  term  "prototype"  to  mean  an  engineering 
model  that  could  be  based  on  commercially  available  parts  instead  of  a  model  built  to 
released  drawings  with  production-worthy  parts  and  processes  [Ref.  22:  p.  53]. 

The  authors  provide  many  reasons  to  demonstrate  the  need  for  a  common 
vocabulary  at  the  project  level  [Ref.  22:  p.  27]: 

-Most  companies  do  not  have  a  common  vocabulary. 

-Schools  do  not  have,  and  consequently  do  not  teach  a  common  vocabulary. 

-Words  are  used  differently  across  projects,  companies,  and  industries. 

-Terminology  manuals,  when  they  exist,  are  often  imprecise. 

-There  is  little  effort  to  fix  the  problem. 

To  alleviate  this  deficiency,  the  authors  recommend  a  terminology  manual  for 
each  project.  A  terminology  manual,  taking  into  consideration  the  terminology 
appropriate  to  the  industry,  company,  and  the  specific  project,  is  indispensable  for 
mitigating  the  communication  problems  caused  by  a  vocabulary  breakdown. 

2.  Teamwork 

Teamwork  is  the  second  perpetual  property  of  projects.  Without  a  commitment  to 
teamwork,  project  activity  is  subject  to  seemingly  chaotic  activities.  In  fact,  the  DoD 
recognized  this  "essential"  by  including  it  as  a  major  theme  in  the  recent  update  to  the 
5000  series  documents. 
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To  help  project  managers  achieve  real  teamwork;  the  authors  provide  five 
fundamentals  of  teamwork  and  the  techniques  for  achieving  them  [Ref.  22:  p.  31]: 

a)  Common  Goals 

The  first  step  to  building  teamwork  is  to  clearly  define  the  group 
objectives.  Managers  need  to  outline  the  roles  and  responsibilities  required  by  the  team 
members  to  accomplish  the  objectives.  Additionally,  managers  must  ensure  that  team 
members  can  articulate  these  common  goals  and  that  they  are  committed  to  achieving 
them.  [Ref.  22:  p.  62] 

b)  Acknowledged  Interdependency  and  Mutual  Respect 

When  interdependencies  are  not  acknowledged,  there  is  no  footing  for 
teamwork.  There  is  only  the  foundation  for  "well  structured  independent  effort."  [Ref.  22: 
p.  63]  For  interdependencies  to  be  recognized,  the  team  members  need  to  accept,  and 
respect,  the  roles  that  must  be  filled  by  each  team  member.  The  authors  list  the  three  steps 
to  acknowledging  interdependencies  and  establishing  expectations  as  [Ref.  22:  p.  63]: 

-Defining  the  specific  functions,  tasks,  and  individual 
responsibilities. 

-Developing  an  organizational  structure  and  defining  team 
interdependencies. 

-Defining  the  scope  of  each  team  member. 

These  steps  are  not  only  recommended  for  use  during  the  initial  formation 
of  the  team,  but  also  for  the  subsequent  inclusion  of  new  team  members. 

c)  A  Common  Code  of  Conduct 

The  most  commonly  overlooked  factor  in  teamwork,  according  to  the 
authors,  is  a  common  code  of  conduct.  Even  though  the  obvious  conduct  issues  are 
usually  well  documented  by  company  and  government  policies,  they  still  may  not  be  well 


22 


known  to  all  the  team  members.  Gray  areas  are  especially  subject  to  misinterpretation  and 
confusion.  Compounding  the  problem,  managers  often  assume  that  a  code  of  conduct  is 
implied  and  understood  even  if  it  has  not  be  specifically  defined  and  agreed  to.  The  end 
result  can  be  the  destruction  of  teamwork  through  tension  and  separation  among  the  team 
members. 


The  authors  provide  guidelines  for  a  code  of  conduct.  To  be  effective,  a 
common  code  of  conduct  must  [Ref.  22:  p.  65]: 

-Establish  rules  of  behavior. 

-Reach  consensus  on  the  definition  of  an  ethical  code  of  conduct. 

-Document  the  most  significant  factors. 

Managers  therefore  have  a  responsibility  to  review  the  issues  to  ensure  that 
all  team  members  are  aware  of  the  potential  problems.  Figure  3-1  illustrates  an  overview 
of  legal  conduct  issues. 

Ethical  issues  are  more  difficult  to  enumerate  than  legal  issues.  Individuals 
usually  rely  on  personal  values  to  navigate  through  conflicts  between  company  practices, 
laws  and  regulations,  and  management  directives.  To  help  team  members  negotiate 
ethical  issues;  the  authors  provide  these  general  guidelines  [Ref.  22:  p.  65]: 

-Seek  higher  management  guidance  to  confirm  difficult  choices  for 
conflicts  among  the  various  codes  of  conduct. 

-If  asked  to  operate  in  a  potentially  improper  manner,  make  sure 
that  the  request  is  written  and  verify  it  with  the  cognizant 
authority. 

-Report  any  improper  conduct,  anonymously  if  necessary. 
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Peddling 


Indirect 


Figure  3-1  From  [Ref.  22:  p  65],  Legal  Conduct  Issues. 

d)  Shared  Rewards 

The  concept  of  shared  rewards  is  simple:  effective  rewards  start  with  fair 
and  equitable  compensation  for  each  position  on  the  team.  Individual  members  or  the 
entire  team  can  earn  rewards.  If  the  project  manager  prefers  to  divide  cash  awards  among 
the  team  members,  the  authors  recommend  the  money  be  given  in  small  amounts  so  more 
can  share.  Another  strategy  offered  is  to  simply  spend  the  money  on  team  recognition. 

e)  Team  Spirit  and  Energy 

This  teamwork  fundamental  is  a  factor  of  personal  attitude  as  well  as 
company  culture.  The  authors  are  careful  to  point  out  [Ref.  22:  p.  67]  that  putting  the 
team  ahead  of  oneself  does  not  mean  that  team  spirit  requires  the  elimination  of 
independent  "pace  setters."  The  key  for  members  with  driven  personalities  is  to  allow 
them  to  exercise  their  assertiveness  and  energy  without  dominating  teammates. 

f)  Techniques  for  Creating  Teamwork 

It  is  the  responsibility  of  the  project  manager  to  draw  on  the  strength  of  the 
team  and  to  neutralize  its  weakness.  To  help  prevent  the  manager  from  micro  managing 
or  providing  too  much  freedom  to  the  team,  the  authors  present  a  list  of  responsibilities 
for  the  manager.  Accordingly,  the  project  manager  must  [Ref.  22:  p.  67]: 
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-Clearly  define  unambiguous  responsibilities. 

-Define  and  communicate  a  project  process  and  style. 

-Delegate  whenever  possible. 

-Empower  the  team  to  be  accountable. 

-Balance  support  with  direction  as  required. 

-Train  the  team,  by  example,  to  operate  as  a  team. 

-Deal  with  under-performers  who  drag  the  team  down. 

-Establish  team-effort  rewards. 

-Design  the  tasks  and  work  packages  in  a  way  to  encourage 
teamwork. 

Finally,  managers  need  to  confirm  that  their  leadership  is  effective  by 
measuring  observable  behavior.  There  are  positive  and  negative  indicators  that  a  manager 
can  observe  to  gauge  the  status  of  the  team  cohesiveness  (Table  3.1)  [Ref.  22:  p.  73]. 


Positive  Indicators 

Negative  Indicators 

A  positive  cooperative  climate  prevails. 

A  climate  of  suspicion  and  mistrust  exists. 

Information  flows  freely  between  team  members. 

Information  is  hoarded  or  withheld. 

No  work  is  considered  beyond  an  individual’s  job 
description. 

Fingerpointing  and  defensiveness  prevails. 

Interpersonal  interactions  are  spontaneous  and 
positive. 

Counterproductive  subgroups  and  cliques  begin  to 
form. 

The  collective  energy  of  the  team  is  high. 

Fear  of  failure  causes  individuals  to  avoid  or 
postpone  making  important  decisions. 

Table  3-1.  Positive  and  Negative  Indicators. 


3.  Project  Cycle 

All  projects  pass  through  a  sequence  of  events  that  the  authors  identify  [Ref.  22: 
p.  32]  as  the  "project  cycle".  Different  organizations  have  their  preferred  project  models 
from  which  they  tailor  to  fit  each  specific  project.  The  project  cycle  is  used  as  a  roadmap 
for  the  project  to  progress  from  "stake-to-stake",  or  from  one  milestone  to  the  next. 

Without  the  use  of  a  field-tested,  clearly  defined  sequence  of  actions  for  a  project 
to  follow,  teams  are  forced  to  create  their  own  "path".  As  a  result,  the  consequences  for 
not  using  a  proactive  framework  for  the  project  can  be  devastating.  [Ref.  22:  p.  76] 
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The  project  cycle  template  described  by  the  authors  is  decomposed  into  many 
components  and  subcomponents.  It  has  Periods,  Phases  within  each  Period,  Control 
Gates,  Activities,  and  Products.  It  also  contains  three  “interwoven”  layers,  the  Technical, 
Business,  and  Budget  aspects  that  occur  in  parallel  within  the  project  cycle.  [Ref.  22:  p. 
32] 

a)  Periods 

The  project  template  is  divided  into  three  periods:  the  Study  Period,  the 
Implementation  Period,  and  the  Operations  Period  (Figure  3-2).  Each  of  these  periods 
corresponds  to  the  three  major  stages  of  a  project  as  it  progresses  from  the  identification 
of  the  user’s  need,  through  concept  determination,  implementation,  and  finally  to 
production  and  user  operation.  [Ref.  22:  p.  77] 

The  study  period  is  cmcial  to  the  project  because  it  is  during  this  time  that  the 
scope  and  funding  is  determined.  Even  though  a  project  team  must  engage  in 
considerable  analysis  and  negotiation  during  the  Study  Period,  a  thorough  study  can  often 
prevent  lost  time  and  wasted  funds  as  the  result  of  rework. 

The  Implementation  period  sets  the  contractual  foundation  for  the  project. 
Objectives  for  this  period  include  choosing  the  highest  value  bidder,  designing  and 
building  the  first  article,  and  testing  and  verifying  the  article  in  accordance  with  all 
specifications. 

It  is  during  the  Operations  Period  that  the  user’s  needs  are  fulfilled  and  the  project 
solution  in  achieved.  In  a  government  project  the  objectives  for  this  period  are:  to  transfer 
the  system  to  the  operational  location  and  establish  full  operational  capability,  operating 
and  maintaining  the  system  according  to  the  user’s  requirements,  and  identifying  system 
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improvements  for  future  use  [Ref.  22:  p.  82],  In  commercial  projects  the  system  is 
delivered  to  users  in  the  marketplace.  Finally,  in  applicable  projects,  planning  for 
deactivation  takes  place. 


Figure  3-2  From  [Ref.  22:  p  33].  The  Sequential  Project  Cycle. 


b)  Control  Gates 

The  authors  define  a  control  gate  as  "a  management  event  important 
enough  to  be  defined  and  included  to  the  schedule  by  executive  management."  [Ref.  22: 
p.  83]  They  are  the  major  decision  points  in  a  project  cycle  that  ensure  the  scheduled 
activities  are  completed  successfully  before  allowing  the  pursuit  of  new  activities.  The 
primary  objectives  of  Control  Gates  are  listed  as  [Ref.  22:  p.  83]: 

-To  ensure  that  all  current  phase  activities  and  products  are 
complete. 

-To  ensure  that  progression  to  the  next  set  of  activities  is  based  on 
hard  evidence  that  the  team  is  prepared  and  that  the  risk  of 
proceeding  is  acceptable. 

-To  promote  a  synergistic  team  approach. 
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c)  Activities 

Activities  are  those  specific  actions  taken  by  the  project  team  to  meet  the 
goals  of  the  project.  Examples  of  these  activities  include  [Ref.  22:  p.  34]: 

-Defining  user  requirements. 

-Comparing  trade-off  candidate  concepts. 

-Developing  a  user  validation  approach. 

d)  Products 

Products  are  the  output  of  activities.  Products  usually  need  to  be  approved 
at  the  Control  Gates.  Examples  of  Products  include  [Ref.  22:  p.  34]: 

-System  Concept  Document. 

-Specifications,  drawings,  and  manuals. 

-Internal  hardware  and  software  feasibility  models. 

-Deliverable  hardware,  software,  and  documentation. 

e)  Budget 

The  budget  aspect  is  one  of  three  layers  to  every  project  cycle  and 
represents  the  activities  and  events  needed  to  provide  the  project  with  the  necessary  funds 
throughout  its  cycle.  The  budget  contains  all  events  pertaining  to  the  executive 
management  appropriation  of  project  funds  and  a  project  manager's  securing  of  funds. 
[Ref.  22:  p.  85] 

f)  Business 

The  business  aspect  contains  all  the  events  related  to  the  overall  business 
management  of  the  project  and  associated  contract  management.  One  important  business 
management  event  is  pursuing  and  managing  customers.  Other  events  include  activities 
needed  to  solicit,  select,  and  manage  the  vendors  who  are  participating  in  the  project. 
[Ref.  22:  p.  86] 
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g)  Technical 

The  combination  of  the  events  from  the  three  project  cycle  aspects 
constitute  the  total  cycle  (Figure  3-3).  The  budget  and  business  activities  are  overlaid  on 
the  technical  aspect  to  form  the  complete  project  cycle  (Figure  3-4).  Like  the  other  two 
layers,  the  technical  aspect  of  the  project  cycle  contains  its  own  set  of  events  even  though 
it  occurs  in  parallel  with  the  other  two  aspects.  [Ref.  22:  p.  33] 

The  technical  cycle  contains  all  the  events  related  to  satisfying  the 
technical  and  quality  requirements  [Ref.  22:  p.  35].  It  identifies  the  activities  and  events 
from  a  system  engineering  perspective  required  to  provide  the  most  effective  and  efficient 
technical  solution  to  the  project  requirements. 

The  actual  process  is  best  visualized  as  “Vee-Model”  rather  than  in  a 
horizontal  format  (Figure  3-5).  The  schedule  maturity  and  time  progress  from  left  to  right 
on  the  model.  On  the  left  side,  one  performs  decomposition  and  definition  while  moving 
from  system  requirements  and  concepts  down  to  fabrication,  assembly,  and  code  to 
"build-to"  documentation.  The  right  side  of  the  Vee  corresponds  to  an  upward  movement, 
aggregating  the  system's  components  at  successively  higher  levels  while  performing 
integration  and  verification.  [Ref.  22:  p.  34] 
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The  PROJECT  CYCLE 


Each  event  or  product  of  an  event ,  is  assigned  to  the  appropriate  category 


These  events  involve  planning  for,  and  securing, 
project  funding  to  fuel  the  project  through  the  cycle. 


Control 

Gates 


Specific  actions  taken  to  meet  the  goafs  of  the  project 
e.g.,  Define  user  requirements. 

Trade-off  candidate  concepts, 

Develop  user  validation  approach. 


The  output  of  activities  -  to  be  approved  at  the  Control  Gate, 
e.g..  System  Concept  Document 

Specifications,  drawings,  and  manuals 

Internal  hardware  and  software  feasibility  models 

Deliverable  hardware,  software,  and  documentation. 


Predetermined  decision  check  points  to  be  satisfied 
before  advancing  to  the  next  set  of  activities, 
e.g..  System  Concept  Review. 


Figure  3-3  From  [Ref.  22:  p.  34].  Total  Cycle  Format. 


Technical  -  contains  all  events  related  to  satisfying 
the  technical  and  quality  requirements. 
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Figure  3-5  From  [Ref.  22:  p.  108].  The  Vee  Model. 

The  utility  of  the  Vee  format  is  in  the  capability  to  better  visualize  the 
direct  correlation  between  the  activities  on  the  left  and  the  right  side  of  the  model.  One 
example  of  this  correlation  is  the  manner  in  which  the  model  minimizes  the  chances  that 
requirements  are  specified  in  a  way  that  cannot  be  measured  or  verified.  While  using  the 
Vee-model,  the  method  of  verification  to  be  used  on  the  right  side  must  first  be 
determined  during  activities  on  the  left  side  before  descending  down  to  the  next  level  on 
the  left  side. 

One  of  the  valuable  uses  for  the  Vee-model  is  as  a  tool  for  controlling  the 
systems  engineering  process,  which  the  authors  define  [Ref.  22:  p.  37]  as  the  application 
of  the  "Systems  Analysis  and  Design"  and  the  "Systems  Integration  and  Verification" 
subprocesses  to  the  technical  aspect  of  the  project  cycle.  These  two  subprocesses  are 
further  defined  in  the  following  Project  Management  Elements  section. 

Another  valuable  use  for  the  Vee-model  is  as  a  tool  for  risk  management. 
The  Vee-model  offered  by  the  authors  encourages  detailed  work  early  in  the  cycle  to 
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reduce  risk.  An  example  [Ref.  22:  p.  88]  is  the  accommodation  of  hardware  and  software 
"requirements-understanding"  and  technical  feasibility  models  in  the  first  period.  Risk 
identification  and  reduction  modeling  continues  through  the  conduct  of  "off-core" 
activities  (Figure  3-6). 

Unlike  decisions  made  during  the  off-core  studies,  only  decisions  made  at 
the  core  of  the  Vee  are  put  under  configuration  management  at  the  appropriate  control 
gates.  These  configurations  comprise  the  baseline  definition  for  the  system  under 
development. 


“Off-Core”  User  Discussions 
and  Approvals 

“Are  the  proposed  solutions  acceptable?” 


Baseline  Being 

^  \  Verification  /  ^ 

Planned 

Considered 

^  “How  to  prove  that  the  solution  ^ 

Verification 

Baseline 


Core  of  the  “Vee’ 

Plans.  Specifications,  and 
Products  Are  Under  Project 
Configuration  Management 


V 


Baselines  to  be 
Considered 

{Core  of  the  ”Vee”) 


Baselines  to  be 
Verified 

{Core  of  the  ‘Vee”) 


“Off-Core”  Risk  and 
Opportunity  Management 
Investigations  and  Actions 

uHow  are  the  risks  and  opportunities  of  the 
proposed  solutions  being  managed?” 


Time  and  Baseline  Maturity 


Figure  3-6  From  [Ref.  22:  p.  89].  Vee  Model  and  Off  Core  Activity. 


4.  Project  Management  Elements 

As  the  authors  note,  the  technical,  schedule  and  cost  performance  elements  of  a 
project  do  not  complement  each  other.  Rather,  they  are  opposing  forces  that  require  a 
skillful  blending  to  reach  the  best  solution  for  the  given  situation.  The  Project 
Management  Elements,  applied  situationally  throughout  the  project  cycle,  provide  the 
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tools  and  techniques  needed  to  keep  the  aforementioned  parameters  in  balance.  [Ref.  22: 
P-39] 


The  Project  Management  Elements  consist  of  10  categories  of  management 
responsibilities,  functions,  techniques,  and  tools  that  help  manage  [Ref.  22:  p.  104]: 

-All  types  of  projects. 

-All  phases  of  the  Project  Cycle. 

-All  organizations  participating  in  the  project. 

a)  Project  Requirements 

The  project  requirements  constitute  requirement  creation  and  management. 
The  major  documents  created  to  support  this  element  are  the  products  referred  to  as 
"outputs"  in  the  project  cycle.  The  framework  for  applying  this  element  is  embodied  by 
the  Vee-model  discussed  in  the  previous  section. 

The  first  technique  for  determining  requirements  is  the  System  Analysis 
and  Design  (SA&D)  process.  It  is  applied  at  every  level  of  the  decomposition  and 
definition  sequence  while  working  down  the  left  side  of  the  Vee-model.  The  SA&D 
process  is  initiated  by  the  higher  level  requirements  and  constrained  by  the  approved 
baselines.  There  are  several  steps  within  the  SA&D  (Figure  3-7)  [Ref.  22:  p.  108]: 

-Define  the  problem  and  evaluation  criteria. 

-Understand  the  context  of  implementation. 

-Define  the  required  behavior  and  performance. 

-Develop  candidate  physical  or  logical  solutions. 

-Select  the  best  solution. 

-Get  customer  approval  of  solution  and  add  to  baseline. 

The  Systems  Integration  and  Verification  (SI&V)  process  is  the 
counterpart  to  SA&D.  It  is  an  iterative  process  that  is  repeated  throughout  the  integration 
and  verification  sequence  while  navigating  up  the  right  side  of  the  Vee-model  format. 
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SI&V  provides  the  framework  for  using  the  verification  criteria  developed  during  the 
decomposition  and  definition  process  while  inspecting  and  testing  the  systems  integration 
that  occurs  at  each  level.  Similar  to  SA&D,  there  are  also  several  steps  within  the  SI&V 
process  (Figure  3-8)  [Ref.  22:  p.  116]: 

-Inspect  and  test  to  verification  requirements. 

-Integrate  with  the  next  configuration  item  and  repeat  verification  process. 

-If  needed,  identify  and  correct  deficiencies. 

-If  needed,  document  uncorrectable  deficiencies. 

-If  needed,  confirm  impact  of  uncorrectable  deficiencies. 

-If  needed,  modify  approved  technical  baseline  to  incorporate  deviation. 

Another  requirements  management  responsibility  is  the  "traceability"  and 
"accountability"  of  project  requirements.  The  purpose  is  to  ensure  that  all  requirements 
have  been  incorporated  into  the  design  and  that  requirement  satisfaction  is  verified  by 
test,  inspection,  demonstration,  and  if  needed,  analysis.  [Ref.  22:  p.  118] 


Higher  level 
requirements  and 
constraints  from 
approved  baseline 


1 


Define  the 
problem  to  be 
solved  and  the 
evaluation 
criteria  for  the 
solutions 

Understand  the 
Concept  of 
Operations  of  the 
level  under 
analysis  as  driven 
by  higher  level 
baselines 

i 

No 


Define  the 
required  behavior! 
In  a  functional 
interaction 
diagram 


|  Candidate  3 
f_  Candidate  2 


Define  the 
required 
functional 
performance  by  | 
quantitative 
analysis 


Candidate  1 

Conceive 

candidate 

solutions 


baa 


Evaluate 

solutions  against 
criteria,  evaluate 
risks  and  failure 
modes  and  select 
the  best  solution 


CD  I 


£ 


Get  buyer 
approval  of 
solution,  as 
required,  and 
Include  solution 
into  the  approved 
technical  baseline 


Output  is  the 
solution 
“Design  to” 
specification  and 
Concept  of 
Operations  for 
the  next  level 


Next  level 
requirements 
and  constraints 
from  approved 
baseline 


Figure  3-7  From  [Ref.  22:  p.  110].  System  Analysis  and  Design  Flow  Chart. 
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Verification  Requirements  and 


Figure  3-8  From  [Ref.  22:  p.  116].  System  Integration  and  Verification  Flow  Chart. 


b)  Organization  Options 

The  proper  organizational  structure  can  contribute  significantly  to  the 
project's  performance  and  efficiency.  Conversely,  over-complexity  or  redundancy  in 
organizational  design  can  lead  to  confused  responsibility  and  inefficiency.  To  find  the 
best  organizational  fit,  it  is  important  to  ensure  that  project  requirements  are  considered 
because  each  project  presents  a  unique  set  of  organizational  requirements.  The  authors 
present  several  organizational  structure  options  and  the  relative  strengths  and  weaknesses 
for  each:  [Ref.  22:  p.  128] 

-Pure  functional:  This  organization  is  rated  as  the  best  for  a  single 
project  that  is  relatively  independent  in  interface  or  technology.  It 
is  rated  as  not  good  for  any  organizations  that  are  managing 
multiple  projects. 

-Pure  project:  This  option  is  rated  as  the  best  choice  for  projects 
that  prioritize  the  schedule  or  cost  performance  as  high  and 
development  costs  as  relatively  unimportant. 

-Conventional  matrix:  This  organization  configuration  works  best 
if  the  project  manager  controls  the  funds  and  has  well  defined 
interrelationships  with  the  supporting  managers.  However,  the 
matrix  fails  in  those  circumstances  where  the  project  manager  is 
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viewed  as  a  "coordinator"  and  the  support  managers  operate  on  a 
"best  effort"  basis. 

-Collocated  Matrix:  Managers  should  consider  this  option  for  very 
high  priority  projects  that  depend  on  critical  resources  and 
technologies,  and  when  on-going  involvement  with  company 
strategy  is  secondary. 

c)  Project  Team 

Team  formation  is  a  situational  process  that  is  ongoing  throughout  the 

project  life-cycle.  This  is  because  the  process  of  forming  the  team  involves  more  than  just 

determining  staffing;  it  also  encompasses  the  definition  and  management  of  interfaces 

with  project  stakeholders  such  as  supporting  organizations,  contractors,  upper 

management,  and  the  customer.  [Ref.  22:  p.  132] 

As  a  part  of  the  team  staffing  strategy,  senior  management  should 

document  job  responsibilities  prior  to  selecting  the  project  manager.  The  authors  state 

[Ref.  22:  p.  134]  that  this  documentation  should  establish  responsibility  for: 

-Establishing  the  team  and  teamwork  environment. 

-Inspiring  and  motivating  the  team. 

-Ensuring  all  project  requirements  are  clearly  defined  and 
documented  and  disseminated  to  the  lowest  level. 

-Ensuring  the  controls  are  in  place  and  effective. 

-Controlling  baseline  requirements  through  a  change  control 
system. 

-Ensuring  that  the  visibility  techniques  are  in  place  and  effective. 
-Determining  the  frequency  and  the  level  of  status  that  is 
appropriate  for  the  project. 

-Timely  execution  of  corrective  action  to  recover  the  project  plan. 

The  project  manager  is  the  key  person  on  the  project  team.  As  the  authors 
have  noted  [Ref.  22:  p.  135],  the  project  manager  must  interface  in  three  different  areas. 
He  or  she  must  meet  the  requirements  of  the  customer,  answer  to  senior  management,  and 
provide  a  positive  work  environment  for  the  project  team.  Therefore,  selecting  the  project 
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manager  is  a  critical  matchmaking  task  for  upper  management  that  is  crucial  to  the 
project's  success.  The  authors  provide  [Ref.  22:  p.  136]  a  sample  "competency  model" 
that  can  be  used  when  selecting  a  team  member  as  an  objective  basis  for  evaluating  the 
key  competency  factors  required  for  a  particular  project.  Figure  3-9  depicts  a  model  that 
is  tailored  for  selecting  a  project  manager. 

The  project  manager  needs  to  determine  the  required  functions  and  related 
skills  as  shown  in  Figure  3-9  before  selecting  team  members.  For  this  task,  the  nature  of 
the  project  will  guide  the  project  manager.  It  is  also  important  to  select  the  core  team 
from  people  who  have  previous  experience  at  the  task  or  project  management  level. 


Rating  Factor 

Basic 

Score 

Advanced 

Score 

Expert 

Score 

Project 

Management 

Training 

Has  had  some  proj¬ 
ect  management 
training 

Has  had  the  com¬ 
pany's  or  equivalent 
project  manage¬ 
ment  training 

Has  had  the  com¬ 
pany's,  PMI1,  or 
equivalent  certifica¬ 
tion  in  project  man¬ 
agement 

Project 

Management 

Experience 

Has  served  as  a 
deputy  or  assistant 
project  manager 

Has  been  a  success¬ 
ful  project  manager 

Has  managed 
several  successful 
programs 

Contracting 

and 

Negotiating 

Is  knowledgeable  of 
types  and  applica¬ 
tions  of  the  relevant 
contract  types 

Has  participated  in 
developing  contract 
negotiation 
strategies 

Has  considerable  ex¬ 
perience  in  contract 
negotiation  strategy 
and  participating  in 
negotiations. 

Sub¬ 

contracting 

Is  knowledgeable  in 
the  difference  be¬ 
tween  purchasing 
and  subcontracting 

Has  participated  in 
the  selection  and 
award  of  subcon¬ 
tracts 

Has  successfully 
managed  subcon¬ 
tractors 

Decision 

Analysis 

Is  aware  of  the  im¬ 
portance  and  prac¬ 
tice  of  Analytical 
Decision  Analysis2 

Has  been  trained  in 
Analytical  Decision 
Analysis2 

Has  been  trained 
and  routinely  prac¬ 
tices  Analytical  De¬ 
cision  Analysis2 

Figure  3-9  From  [Ref.  22:  p.  136].  Example  Competency  Model. 


In  addition  to  the  project  manager,  there  are  two  other  important  positions 
that  make  up  the  core  of  the  team:  the  systems  engineer  and  the  business  manager.  The 
systems  engineer  is  responsible  for  the  project's  technical  integrity  while  meeting  the  cost 
and  performance  objectives.  The  business  manager  is  responsible  for  all  the  business 


37 


aspects  of  the  project  to  include  planning,  scheduling,  and  contract  matters.  [Ref.  22:  p. 
139] 


d)  Project  Planning 

The  authors  define  planning  as  "the  process  which  determines  beforehand 
the  activities  necessary  to  complete  the  project."  [Ref.  22:  p.  148]  At  the  total  project 
level,  planning  is  performed  in  each  project  cycle  phase.  As  a  result,  the  overall  plan  is 
composed  of  separate  plans  for  each  period  and  phase  of  the  project  (Figure  3-10).  At  the 
most  basic  level,  the  plan  contains  [Ref.  22:  p.  149]: 

-What  is  to  be  done. 

-When  it  should  be  done. 

-Who  is  responsible  for  doing  it 


Figure  3-10  From  [Ref.  22:  p.  149].  The  Total  Project  Plan. 

The  goal  of  planning  parallels  the  goal  of  the  project:  ensure  that  the 
commitments  made  to  the  customer  are  met.  To  determine  the  project  deliverables,  and 
provide  a  description  of  each,  the  authors  recommend  [Ref.  22:  p.  155]  that  project 
managers  use  the  Project  Product  List  (PPL)  and  Fact  Sheets.  The  PPL  and  Fact  Sheets 


are  derived  from  system  decomposition  and  definition  and  are  a  listing  of  all  contract 
deliverables,  and  their  quantities  required. 

The  Work  Breakdown  Structure  (WBS)  is  a  method  used  to  decompose  a 
system  by  assemblies,  subassemblies,  and  components  instead  of  by  functional 
organization  or  discipline  (Figure  3-11).  The  WBS  is  essential  for  project  planning 
because  it  becomes  the  basis  for  work  assignments,  budgeting,  scheduling,  risk 
assessment,  cost  collection,  and  performance  statusing.  The  authors  offer  the  following 
guidance  when  preparing  a  WBS  for  hardware  or  software  related  project  [Ref.  22:  p. 
158]: 


-Structure  the  WBS  by  product  and  elements  of  the  product. 

-Include  all  authorized  tasks. 

-Place  cost  collection  one  level  below  budget  performance  report 
to  facilitate  problem  cause  identification. 

-Ensure  identifiers  for  like  tasks  are  similar. 

-Collect  all  tasks  for  an  element  with  the  element  identifier. 

-WBS  depth  depends  on  the  risk  to  be  managed  and  reported. 

The  WBS  provides  an  essential  foundation  for  the  project  network  and  for 
scheduling.  To  create  this  foundation,  the  tasks  devised  from  the  WBS  are  combined  to 
form  the  project  network  from  which  the  scheduling  will  flow  (Figure  3-12). 

While  there  are  other  methods  such  as  the  Project  Evaluation  and  Review 
Techniques  (PERT)  and  the  Critical  Path  Method  (CPM)  for  developing  a  schedule,  the 
authors  promote  their  own  process  called  "cards-on-the-wall"  (COW)  [Ref.  22:  p.  163]. 
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System  Components 
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“glue”  (integration) 


Comp 

onent  A 
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B 
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C 
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D 

The  whole  does 
more  than  the 
sum  of  its  parts 


Product 

Breakdown  Structure 
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Structure  (WBS) 

All  work  components 
necessary  to  produce 
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■■■■ 

Design  Fab  Assemble  Test! 


■■■■ 
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i"  1  i 

System  Project 
Engineering  Management 


Figure  3-11  From  [Ref.  22:  p.  159].  The  Work  Breakdown  Structure. 
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Figure  3-12  From  [Ref.  22:  p.  162].  The  Work  Breakdown  Structure  Tasks  Related  to 

the  Project  Network  and  Schedule. 


The  drawback  to  PERT  and  CPM  is  that  they  allow  a  single  person,  using  automated 
tools,  to  complete  the  construction  of  the  schedule.  However,  the  COW  method  facilitates 
the  interaction  of  the  team  because  the  members  post  the  separate  work  packages,  by 
project  phase,  on  the  wall  for  all  to  see.  This  method  allows  all  of  the  members  of  the 
.  team  to  congregate  and  provide  input  to  the  planning  process  as  well  as  exposing  them  to 
the  interdependencies  of  each  work  package. 

Using  the  COW  method,  the  authors  present  several  steps  from  which  the 
scheduling  process  iterates  [Ref.  22:  p.  162]: 

-Combine  the  tasks  to  form  a  project  network. 

-Define  and  evaluate  risks. 

-Develop  risk  mitigation  actions  and  add  to  the  network. 

-Factor  in  task  duration  time. 

-Determine  the  critical  path. 
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-Shorten  the  critical  path. 

-Commit  to  performing  to  the  task  schedules. 

Devising  a  solid,  coherent  plan  provides  many  benefits  to  the  project 
manager.  Interfacing  with  the  other  stakeholders,  obtaining  resource  commitments,  and 
reacting  to  unforeseen  circumstances  becomes  easier  because  of  the  lack  of  ambiguity.  In 
addition  to  the  techniques  mentioned  in  this  section,  the  authors  provide  [Ref.  22:  p.  153] 
a  table  of  major  elements  and  techniques  for  planning  (Figure3-13). 


Key  Element 
Products 


Tasks 

Strategy 


Network 


Schedules 


Resources 


Process 


Primary  Technique 


Decomposing  the  deliverables  into  their  hierarchi-  Project  Product  List 
cal  structure — all  the  way  from  the  project  require-  and  Fact  Sheets 
ments  down  to  the  lowest  level  internal  and 
external  deliverables. 


Defining  the  tasks  needed  to  complete  each  deliv-  Work  Breakdown 
erable.  Structure 

Identifying  the  risks  and  opportunities  and  the  Lessons  Learned 

customer-compatible  risk  and  opportunity  strategy 
with  preventive,  causative,  and  contingent  action 
plans. 


Logically  arranging  the  required  tasks  to  portray 
the  best  delivery  approach. 


Cards-on-the-Wall  net¬ 
work,  followed  by  a 
computerized  network 
and  critical  path  deter¬ 
mination. 


Scheduling  each  task  then  refining  and  shortening  Scheduling  software 

the  project's  overall  critical  path  through  iterative 

steps. 

Establishing  resources  (personnel,  equipment.  Spread  sheets  and  cost 
finances)  needed  to  accomplish  each  task  on  estimating  models 

schedule. 


Commitments  Committing  the  necessary  resources  and  funds  for  Project  Work  Authoriz- 
each  task  as  determined  from  the  schedule  and  task  ing  Agreements 
definition. 


Figure  3-13  From  [Ref.  22:  p.  153].  Major  Elements  and  Techniques  for  Planning. 
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e)  Risks  and  Opportunities 

Risk  and  opportunity  management  is  a  critical  part  of  the  planning  process 
that  is  performed  simultaneous  with  the  project  plan,  however,  it  uses  a  separate  and 
unique  process.  The  authors  provide  specific  steps  for  risk  and  opportunity  management 
to  allow  a  project  manager  to  make  conscious,  distinct  arguments  for  management 
actions.  This  structured  process  is  ongoing  and  applied  to  the  project  as  it  evolves  [Ref. 
22:  p.  176]: 


-Identify  the  risks  and  opportunities. 

-Assess  both  probability  and  impact  and  then  forecast  the  outcome. 

-Compare  the  outcomes  and  prioritize. 

-Develop  feasible  management  actions  to  enhance  opportunities 
and  mitigate  risks. 

-Estimate  the  cost  of  proposed  immediate  and  contingent  actions. 

-Compare  resultant  changes  to  weighted  value  against  action  costs. 

-Decide  on  actions  required,  and  obtain  concurrence. 

-Document  and  incorporate  decisions  in  all  planning. 

There  are  several  methods  available  to  help  identify  risks  and 
opportunities.  They  range  from  off-core  studies  to  scenario  planning  and  failure  mode 
analysis.  While  the  techniques  vary,  the  common  principle  is  the  same:  to  properly 
identify  the  risks  and  ppportunities,  the  project  team  must  systematically  apply  an 
appropriate  method  through  each  phase  of  the  cycle.  [Ref.  22:  p.  174] 

The  Weighted  Value  (WV)  method  provides  a  good  tool  for  the  project 
manager  to  assess  the  probability  and  the  impact  of  risk  and  opportunity.  The  WV  is 
relatively  simple  to  quantify.  It  is  the  probability  of  occurrence  multiplied  by  the  cost  of 
the  impact  of  that  event  occurring.  For  example,  if  the  probability  of  an  event  occurring  is 
.8,  and  the  impact  of  that  occurrence  is  $100,000,  then  the  weighted  value  for  that  risk  is 
$80,000.  [Ref.  22:  p.  182] 
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Quantifiable  measures  permit  a  project  manager  to  continue  with  the  risk 
and  opportunity  management  process.  The  identified  and  quantified  WV  calculations 
allow  the  project  manager  to  compare  and  prioritize  events.  Armed  with  a  prioritized  list 
of  risk  and  opportunity  events,  management  can  then  make  a  plan  for  influencing  either 
the  probability  of  occurrence  or  the  impact  on  the  outcome  through  the  application  of 
mitigating  or  enhancing  activities.  Once  this  process  is  complete,  the  mitigating  actions 
can  be  incorporated  into  the  project  plan. 

f)  Project  Control 

Project  control  is  process  control.  Further,  process  controls  are  needed  at 
every  level  and  in  each  project  activity  because  projects  without  adequate  process  control 
usually  fail.  The  authors  present  a  dual  system  of  process  control  designed  for  reducing 
risk  [Ref.  22:  p.  191]: 

-Baseline  Control:  Proactive  control  of  the  Project  Plan  and 
changes  to  the  plan  to  help  ensure  that  events  happened  as 
planned,  and  that  events  not  planned  do  not  happen. 

-Performance  Control:  Reactive  control  of  variances  in  the 
performance  of  the  Project  Plan  or  corrective  action  taken  when 
unplanned  events  do  happen. 

For  maximum  effectiveness,  controls  need  to  be  tailored  to  the  project  and 

in  place  before  it  begins.  Proactive  and  reactive  measures  should  be  relevant,  efficient, 

simple,  and  timely.  To  achieve  these  goals,  the  authors  list  five  essential  considerations  to 

a  control  process  [Ref.  22:  p.  191]: 

-Things  to  be  controlled:  The  function  that  must  be  controlled  to  a 
standard  of  performance. 

-Control  Standard:  The  acceptable  standard  of  performance. 

-Control  Authority:  The  person  or  organization  that  imposes  the 
standard  and  can  grant  exceptions. 
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-Control  Mechanisms:  The  forum  or  device  that  measures  and 
controls  conformance  to  the  standard. 

-Variance  Detection:  The  identification  of  deviations  of  the  control 
process  or  violations  of  the  standard. 

An  example  of  process  control  is  the  application  of  these  considerations  to 
"schedule  control."  In  this  example  the  schedule  is  the  thing  to  be  controlled,  the  master 
schedule  is  the  standard,  the  business  manager  is  the  authority,  the  change  board  is  the 
mechanism,  and  the  status  review  provides  the  variance  detection.  [Ref.  22:  p.  43] 

g)  Project  Visibility 

Project  visibility  is  the  means  by  which  the  project  manager  and  project 
team  know  the  status  of  their  project  activities.  Project  visibility  techniques  change  as  the 
project  advances  through  the  project  cycle.  They  are  determined  by  the  timing,  critical 
need,  and  geographic  location  of  the  required  data.  There  are  several  techniques  presented 
by  the  authors  to  gain  visibility  for  project  activities.  Among  these  methods  are  meetings, 
reports,  tiger  teams,  and  glance  management.  [Ref.  22:  p.  212] 

Glance  management  includes  any  number  of  informal  follow-up 
techniques  such  as  "management-by-walking-around"  that  provides  awareness  of  a 
project's  status  with  a  quick  glance.  Other  examples  of  this  type  include  sampling  work  in 
progress,  sitting  in  on  lower  level  meetings  and  engaging  in  conversations  before  or  after 
meetings.  [Ref.  22:  p.  213] 

Tiger  teams  are  used  to  focus  on  trouble  areas.  Members  of  these  teams 
are  usually  technical  experts  or  experienced  troubleshooters  that  can  provide  an  objective, 
third  party  perspective  of  the  problem.  The  ultimate  objective  of  a  tiger  team  is  to  identify 
the  cause  of  a  problem  without  trying  to  assess  the  blame.  [Ref.  22:  p.  216] 
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h)  Project  Status 

Project  statusing  is  the  measurement  of  the  progress  of  the  project  against 
the  project  plan.  The  goal  is  to  identify  variances  that  require  corrective  action  to  restore 
the  plan.  The  authors  contend  that  an  effective  process  for  statusing  [Ref.  22:  p.  224]: 

-Collects  essential  information  only. 

-Measures  primary  variables  that  impact  results. 

-Tailors  information  to  the  needs  of  the  team  members  interpreting 
it. 

Project  managers  should  always  know  the  status  of  four  primary  factors: 
schedule,  technical,  cost,  and  business.  Table  3-2  provides  the  key  metrics  for  each  of 
these  factors.  [Ref.  22:  p.  225] 


Schedule 

Technical 

Cost 

Business 

Progress  summary 

Development  results 

Actual  versus  budget 

Contract  change  process 

Master  schedule 

Design  release 

Headcount 

Actions  to/from  customers 

Milestone 

accomplishments 

Technical  review 

Earned  value  vs. 
expenditures 

Action  to/from  management 

Assemblies  and 
modules 

Technical  performance 
measurements 

Bum  rate  and  overtime 
ratio 

Actions  to/from  contractors 

Tasks 

Interface  control 

Estimate  to  completion 

Funding 

Subcontractors 

Quality 

Estimate  at  completion 

Top-ten  problems 

Parts  and  material 

Profit 

Security  clearances 

Earned  value 

Dispersion  ratio 

Project  manager's  assessment 

Table  3-2.  Status  Factors  and  their  Key  Elements. 


The  authors  provide  several  examples  of  methods  to  measure  the  status  of 
these  key  factors.  One  of  these  methods  is  to  use  the  quantity  of  milestones  as  a  schedule 
metric  (Figure  3-14)  [Ref.  22:  p.  228].  Another  example  provided  is  a  Total  and 
Experienced  Headcount  variance  report  to  anticipate  cost  and  schedule  problems  (Figure 
3-15)  [Ref.  22:  p.  229]. 


46 


Number  of  Milestones 


■■  Planned 
■  ■  Actual 


Figure  3-14  From  (Ref.  22:  p.  231].  Headcount  Variance  Report. 


Figure  3-15  From  [Ref.  22:  p.  228].  Milestone  Status  Report. 
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One  of  the  more  prevalent  systems  used  today  to  refine  performance 
measurements  is  the  Earned  Value  (EV)  method.  While  the  EV  method  requires  detailed 
planning  to  establish  and  maintain,  it  provides  a  manager  with  a  powerful  tool  to  provide 
early  warning  of  problems  by  relating  cost,  schedule  and  technical  progress.  Figure  3-16 
illustrates  an  application  of  the  EV  method  to  a  project.  [Ref.  22:  p.  233] 

In  this  example,  the  metrics  denotes  that  the  project  depicted  in  Figure  3- 
16  is  behind  schedule  and  over  budget.  The  variance  between  the  Budgeted  Cost  of  Work 
Performed  (BCWP)  and  the  Actual  Cost  of  Work  Performed  (ACWP)  signifies  that  the 
actual  costs  of  completing  the  tasks  exceeded  the  budgeted  costs  and  therefore  the  project 
has  expended  more  funds  than  anticipated  at  that  point  in  the  schedule.  The  variance 
between  the  BCWP  and  the  Budgeted  Cost  of  Work  Scheduled  (BCWS)  connotes  that  the 
project  has  performed  less  work  than  scheduled.  Through  comparisons  of  these  and  other 
metrics,  the  project  manager  can  interpret  trends  and  anticipate  the  eventual  cost  of  the 
completed  project. 


Figure  3-16  From  [Ref.  22:  p.  233].  Earned  Value  Chart. 


i)  Corrective  A  ction 

Corrective  actions  are  those  steps  management  takes  to  restore  the  project 
to  the  plan.  These  actions  are  usually  triggered  by  unacceptable  variances  discovered 
through  the  use  of  statusing  techniques.  By  monitoring  variance  and  recognizing  an 
imminent  or  actual  breach  of  a  predetermined  threshold,  a  manager  maintains  the 
possibility  of  regaining  lost  control  over  a  project.  By  taking  corrective  action  in  a  timely 
manner,  a  manager  can  restore  control  of  the  project. 

According  to  the  authors,  the  steps  for  corrective  are  as  follows  [Ref.  22: 


P-  241]: 


-Analyze  the  problem,  its  current  impact,  and  the  growth  of  the 
impact  if  no  action  is  taken. 

-Prioritize  project  problems  from  the  most  to  the  least  serious. 

-Determine  the  best  approach  using  analytical  decision  analysis. 

Examples  of  corrective  actions  are  overtime,  added  work  shifts,  or  an 

alternate  technical  approach.  Figure  3-17  illustrates  the  use  of  an  analytical  tool  to  help 

evaluate  alternatives  for  corrective  action  that  requires  adjusting  the  work  shifts. 


Evaluation 

Criteria 


Alternative  1 
One  12  hr  shift 


Alternative  2 
Two  8  hr  shifts 


Alternative  3 
iThree  8  hr  shifts! 


Alternative  4 
Two  12  hr  shifts! 


Musts  (Go-No  Go): 

Certified  Software  Testers 

Available  within  3  weeks 


Wants 


Weight 

(W) 


Comments 


Score 

S'  5 
<r 


(Comments 


Score 

i'5 

cc 


Comments 
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cc 


Comments 
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« 
cc 


Factors 
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productivity 

Highly  experienced 
in  our  software 

Low  average  labor 
rate 


10 

8 

8 


10]  80 


|100| 

64 

40 
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260 


186 


214 


204] 


Figure  3-17  From  [Ref.  22:  p.  244].  Evaluation  by  Weighted  Scoring. 
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j)  Project  Leadership 

According  to  the  authors  [Ref.  22:  p.  44],  Project  Leadership  is  the  most 
important  of  the  10  project  management  elements.  Project  Leadership  is  the  bonding 
agent  that  holds  the  previous  nine  project  management  elements  together.  It  is  through  the 
proper  exercise  of  leadership  that  the  other  nine  elements  are  used.  Three  aspects  of 
project  leadership  are  addressed  [Ref.  22:  p.  246]: 

-Situational  leadership  model,  or  the  relationship  of  leadership  to 
management. 

-Techniques  for  inspiring  and  motivating  individual  and  group 
performance. 

-Style,  or  determining  and  communicating  your  leadership  style. 

To  portray  the  situational  nature  of  leadership,  the  authors  use  an 
orthogonal  model  (Figure  3-18).  The  cylinder  represents  the  sequential  actions  for  a 
typical  task.  The  disk  represents  the  project  management  elements  held  together  by  the 
rim  of  project  leadership.  The  orthogonal  positioning  of  the  techniques  of  leadership 
represent  their  situational  application  to  the  project  relative  to  the  project  cycle  and 
current  circumstances. 

To  motivate  the  project  team,  project  managers  should  apply  several 
techniques.  They  need  to  communicate  their  vision  for  success  to  the  project  team.  They 
must  also  create  the  environment  to  enable  the  team  member  to  achieve  the  stated  vision 
or  goals  and  recognize  team  members  for  their  good  performance.  Project  managers 
especially  need  to  draw  on  their  subordinates'  strengths  as  well  minimizing  their 
weaknesses.  Finally,  project  managers  need  to  set  the  example  by  acting  as  they  want  the 
team  to  act. 
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Figure  3-18  From  [Ref.  22:  p.  250].  Orthogonal  Project  Management  Model. 

D.  CHAPTER  SUMMARY 

The  program  management  model  is  critical  to  the  DoD  Acquisition  system 
because  it  provides  a  single  point  of  contact  that  acts  as  an  "integrator"  for  a  complex 
system  of  related  functional  disciplines. 

The  criticality  and  complexity  of  program  management  makes  it  an  ideal  topic  for 
which  to  develop  an  automated  program  that  incorporates  a  qualitative  review  capability 
currently  lacking  within  the  DAU  course  offerings.  The  remainder  of  this  thesis  focuses 
on  developing  such  a  program,  which  will  be  referred  to  as  an  "Automated  Tool  for 
Acquisition  Program  Management  Students"  (ATAPMS). 

The  next  chapter  details  the  content,  structure,  and  topic  requirements  for  an 
ATAPMS  model  tailored  to  the  needs  of  instructors  at  the  NPS. 
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IV.  STRUCTURE  AND  CONTENT  REQUIREMENTS 

A.  INTRODUCTION 

This  chapter  develops  the  structure,  strategies  and  topics  for  an  ATAPMS  tailored 
to  the  NPS.  Because  the  Vee-model  described  in  chapter  three  is  a  useful  concept,  this 
thesis  uses  its  requirements  definition  principles  as  a  guide  to  help  determine  the 
requirements  and  specifications  for  an  ATAPMS  model. 

B.  STRUCTURE 

The  structure  defines  the  top-level  requirements  for  the  application  of  an 
ATAPMS  tailored  to  the  NPS.  The  first  step  for  determining  the  structure  corresponds  to 
the  first  step  in  the  Vee-model:  understand  the  user’s  requirements  [Ref.  22:  p.  36]. 
Feeding  into  the  user  requirements  and  system  concept  phase  of  the  Vee-model  is  the 
“Statement  of  User’s  Needs”  [Ref.  22:  p.  187].  In  the  case  of  this  application  of  an 
ATAPMS,  the  NPS  requires  an  automated  program  to  use  in  a  laboratory  setting  to  assist 
with  the  instruction  of  Acquisition  Program  Management.  The  users  for  this  ATAPMS 
are  identified  as  the  instructors  in  the  Systems  Management  Department  of  the  NPS  who 
teach  Acquisition  Program  Management  related  courses.  Personal  interviews  conducted 
with  six  of  these  faculty  members  verified  several  of  the  important  characteristics 
necessary  to  determining  the  overall  structure  of  an  NPS  ATAPMS. 

While  there  were  several  areas  of  concern  relating  to  the  overall  structure,  based 
on  the  interviews,  the  most  relevant  to  an  ATAPMS  are  the  approach,  setting,  scope, 
scalability,  and  distance  learning  considerations. 
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1.  Approach 

There  are  two  basic  approaches  evident  in  the  automated  tools  used  by  the  DAU 
consortium  schools  to  provide  instruction  to  students:  tutorial  and  capstone. 

The  tutorial  approach  is  designed  to  expose  the  student  to  new  information.  This 
is  the  approach  currently  used  by  the  DSMC  in  introductory  courses  like  Acquisition  101 
that  are  accessed  through  the  Internet.  [Ref.  23:  p.  1] 

The  other  method  is  the  capstone  approach  exemplified  by  the  SAFE  application. 
This  method  does  not  try  to  teach  the  student  new  material,  instead  it  reinforces  or 
integrates  concepts  that  the  students  learned  as  a  prerequisite  to  participating  in  the  class. 
The  capstone  approach  reinforces  these  concepts  by  requiring  the  students  to  apply 
learned  concepts  to  the  problems  presented  by  a  real  world  scenario. 

The  NPS  ATAPMS  should  follow  the  capstone  approach  because  according  to  the 
instructors  interviewed,  there  is  a  greater  need  for  that  type  of  application  than  for  the 
tutorial  method.  In  fact,  other  than  the  SAFE  program,  there  are  no  capstone  type 
automated  programs  used  at  the  NPS  or  within  the  DAU  consortium  to  assist  with  the 
instruction  of  Acquisition  Program  Management. 

2.  Scope 

Because  the  NPS  instructors  will  be  the  primary  users,  the  ATAPMS  should 
accommodate  the  NPS  quarterly  course  class  length.  Courses  are  taught  at  the  NPS  on  a 
quarterly  schedule  with  each  quarter  12  weeks  in  length.  Generally,  the  first  and  last 
weeks  are  not  conducive  to  conducting  laboratory  exercises,  so  the  scope  of  the 
ATAPMS  should  be  to  accommodate  10  weekly  sessions. 
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3.  Setting 

There  are  three  different  scenarios  likely  for  students  using  an  ATAPMS.  Two  of 
these  scenarios  involve  the  resident  classroom  laboratory  environment  and  one  involves 
an  ATAPMS  used  in  a  non-resident  distance  learning  capacity. 

a)  Resident  Students 

One  of  the  classroom  scenarios  involves  accessing  the  module  for  the  first 
time  beginning  at  the  start  of  that  week’s  laboratory  session,  and  the  other  involves 
students  accessing  the  module’s  material  a  day  or  more  prior  to  the  start  of  the  lab 
session.  The  NPS  ATAPMS  should  be  designed  to  incorporate  the  latter  setting.  The 
advantage  to  allowing  students  to  access  materials  before  the  requirements  are  due  is  that 
the  students  will  have  more  time  and  opportunity  to  work  with  the  material.  This 
additional  time  will  also  allow  for  the  inclusion  of  more  detailed  scenarios  and  should 
ultimately  translate  to  a  more  meaningful  learning  experience. 

b)  Non-Resident  Students 

The  second  scenario  envisions  the  use  of  the  ATAPMS  to  support  distance 
learning.  A  program  designed  with  more  involved  scenarios  and  thought  provoking 
material  for  resident  students  will  accommodate  non-resident  students  as  well.  Designing 
the  ATAPMS  to  include  more  detailed  material  allows  for  an  easier  migration  from  the 
traditional  resident  lab  sessions  to  distance  learning  sessions.  This  is  in  part  because 
detailed  course  material  will  naturally  support  the  non-resident  students  enabling  them  to 
work  more  autonomously. 
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4.  Scalability 

One  theme  evident  from  the  research  is  that  there  does  not  appear  to  be  any  one 
specific  design  solution  to  fit  the  acquisition  academic  communities’  requirements.  Of  the 
six  instructors  interviewed,  each  had  subject  matter  topics  and  strategies  for  presenting 
those  topics  that  were  unique  to  their  needs.  They  also  had  students  with  a  variety  of 
educational  backgrounds  who  were  pursuing  different  tracks  in  the  acquisition  field. 

To  be  useful,  an  ATAPMS  must  be  flexible  enough  to  conform  to  each 
instructor’s  needs.  One  method  of  accomplishing  this  flexibility  is  to  design  the 
ATAPMS  using  a  modular  architecture  that  allows  the  replacement  or  addition  of  any  one 
module  without  affecting  the  rest  of  the  program.  With  this  design  format,  each  week’s 
session  would  be  a  stand-alone  module  providing  the  instructors  a  “plug  and  play” 
capability.  Given  this  flexible  format,  instructors  would  have  the  ability  to  select  the 
appropriate  modules  and  present  them  in  the  most  appropriate  order  to  best  tailor  their 
learning  objectives  to  their  category  of  students.  [Ref.  24] 

Another  advantage  to  the  modular  structure  is  the  ability  to  add  new  modules  on 
an  as  needed  base  with  minimal  cost  or  disruption  to  the  existing  program  [Ref.  24],  This 
feature  will  allow  an  ATAPMS  to  stay  relevant  and  to  continue  to  serve  the  academic 
classroom  by  adjusting  to  changes  in  the  acquisition  field.  In  contrast,  the  SAFE  program 
discussed  in  a  previous  chapter  does  not  share  these  advantages,  and  its  inflexibility  is  a 
common  complaint  shared  by  a  number  of  the  instructors  interviewed. 
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5.  Distance  Learning  Considerations 

Even  though  the  current  users  for  the  ATAPMS  are  resident  students,  the  intent  of 
EO  13111  and  the  Defense  Reform  Initiative  are  clear:  DoD  schools  need  to  take 
advantage  of  technology  to  help  make  education  more  accessible  and  at  less  cost. 

Several  of  the  schools  within  the  DAU  consortium  already  use  distance  learning 
technology  to  some  extent.  NPS  has  provided  video  teleconferencing  courses  for  several 
years  [Ref.  25].  The  Defense  Systems  Management  College  [Ref.  23]  and  the  Information 
Resources  Management  College  [Ref.  26]  are  just  two  of  the  DAU  consortium  members 
that  provide  computer  based  distance  learning  courses.  To  insure  that  the  ATAPMS 
meets  Presidential  and  DoD  intentions  and  that  it  is  capable  of  supporting  the  distance 
learning  requirements  of  the  future,  it  should  be  compliant  with  Instructional 
Management  Systems  standards  and  be  designed  to  meet  the  end-state  goals  of  the 
Advanced  Distance  Learning  Initiative. 

The  vision  of  the  ADL  initiative  is  to  “provide  access  to  the  highest  quality 
education  and  training,  tailored  to  the  individual’s  needs,  anywhere  and  anytime  in  the 
world.”[Ref.  27]  The  ADL  initiative  attempts  to  achieve  this  vision  by  transitioning  from 
the  current  model  of  training  which  is  classroom  centered  to  a  model  that  is  network 
centered.  To  foster  the  network  centered  model,  the  ADL  initiative  established  several 
goals  that  a  distance  learning  application  should  meet.  These  end-state  goals  are: 
accessibility,  interoperability,  durability,  re-usability,  and  cost  effectiveness.  [Ref.  28] 
a)  Accessibility 

Accessibility  refers  to  the  student’s  ability  to  have  access  to  the  course 
modules  from  a  variety  of  remote  locations.  Ensuring  modules  are  designed  with  the 
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capability  to  be  accessible  through  the  Internet  is  the  current  method  of  meeting  this  goal 
[Ref.  15]. 

b)  Interoperability 

Interoperability  refers  to  the  student’s  ability  to  use  a  module  at  his 
location  even  though  the  module  was  developed  at  a  different  location  on  a  different 
platform  or  with  a  different  set  of  tools.  Current  indications  from  the  software  industry 
are  that  an  object-based  approach  will  provide  the  basis  for  this  platform  neutral  goal 
[Ref.  16]. 

c)  Durability 

A  durable  module  is  a  module  that  is  robust.  It  requires  no  re-work  or  re¬ 
coding  when  the  base  technology  changes. 

d)  Re-usability 

Re-usability  refers  to  the  capability  of  the  module  to  be  used  in  multiple 
applications.  The  benefit  will  be  a  module  that  saves  development  time  and  money.  The 
same  object  oriented  approach  necessary  for  interoperability  also  provides  the  basis  for 
re-usability  [Ref.  13]. 

6.  Baseline  Summary 

Using  the  Vee-model  method,  it  is  now  possible  to  establish  a  baseline  for  the 
structure  of  the  NPS  ATAPMS  before  determining  the  content.  The  following  list 
summarizes  the  proposed  requirements  for  the  ATAPMS’  top-level  structure: 

-Use  a  capstone  style  approach  to  the  content. 

-Accommodate  at  least  10  weekly  topic  modules. 

-Capable  of  presenting  well  developed  thought-provoking  scenarios  and 
questions. 

-Ensure  modular  architecture  for  scalability  and  a  plug  and  play  capability. 
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-IMS  compliant. 

-Ensure  Accessibility. 

-Ensure  Durability. 

-Ensure  Re-usability. 

-Ensure  Cost  effectiveness. 

C.  CONTENT 

Having  determined  the  structural  requirements,  it  is  now  possible  to  develop  a  set 
of  top-level  content  requirements.  Because  the  research  revealed  that  the  content 
requirements  were  so  varied,  and  the  proposed  structure  is  flexible  enough  to 
accommodate  them  all,  there  is  no  need  at  this  time  to  propose  a  final  solution  set  for  the 
content  requirements.  Therefore,  it  is  not  within  the  scope  of  this  thesis  to  attempt  to 
mature  the  content  requirements  to  the  point  of  recommending  them  as  the  approved 
baseline.  The  content  proposed  in  this  section  will  address  the  most  recurring  content 
needs  as  revealed  by  the  research,  and  subsequent  sections  of  this  thesis  will  draw  from 
this  content  for  the  purpose  of  off-core  prototyping  as  a  proof  of  concept  measure. 

Through  the  course  of  the  research,  it  became  evident  that  the  content  consisted  of 
two  distinct  parts:  topics  and  strategies  for  presenting  the  topics.  An  analysis  of  the 
literature  review  and  instructor  interviews  revealed  that  material  for  instructing 
Acquisition  Program  Management  can  easily  be  grouped  or  categorized  by  logical  topics. 
However,  depending  on  the  objectives  for  a  particular  course,  the  material  included  or 
emphasized  within  the  topic  presented  to  the  student  may  be  different.  For  example, 
Acquisition  Strategy  is  a  topic  that  equates  to  the  Project  Cycle  described  in  chapter  three. 
The  material  related  to  timeline  and  deliverable  expectations  used  in  this  topic  will  differ 
if  the  objective  is  to  emphasize  a  Major  Automation  Information  System  instead  of  a 
Major  Defense  Acquisition  Program  [Ref.  29].  The  following  sections  present  a  listing  of 
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topics  and  strategies  that  are  not  meant  to  be  all-inclusive,  but  they  are  representative  of 
the  recurring  information  derived  from  the  literature  review  and  instructor  interviews. 

1.  Topics 

The  explanation  of  each  topic  presented  in  this  section  will  present  a  higher  level, 

more  generalized  description  because  each  topic’s  content  is  influenced  by  the 

relationship  between  the  topic  and  the  presentation  strategy.  The  intent  of  this  section  is 

to  refine  the  major  topics  within  Acquisition  Program  Management  only  to  the  level 

necessary  to  develop  a  proof  of  concept  prototype  program  in  the  subsequent  sections. 

This  development  strategy  is  consistent  with  the  off-core  activity  method  presented  in  the 

Vee-model  concept  that  states  that  “off-core  activities  do  not  seek  a  final  solution,  but 

rather  a  demonstration  that  one  is  feasible.”  [Ref.  22:  p.  188] 

When  reviewing  the  list  of  candidate  topics,  the  lack  of  topics  related  to 

Acquisition  Reform  themes  is  conspicuous.  They  are  not  listed  as  individual  topics 

because,  in  the  words  of  former  Secretary  of  Defense  William  Perry: 

These  policies  are  fundamental  to  all  of  our  acquisition  activities  essential 
to  clearly  defining  our  performance  requirements,  establishing  affordable 
life-cycle  cost  requirements,  and  working  together  to  develop  affordable 
and  executable  strategies  to  resolve  issues  in  a  timely  manner.  [Ref.  30] 

Using  this  principle  as  a  guide,  there  is  no  need  to  include  the  reform  themes  as 

separate  topics,  instead  each  of  the  following  topics  should  incorporate  the  reform  themes 

into  its  content. 
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a)  Acquisition  Strategy  for  a  Major  Defense  Acquisition  Program 

(MDAP) 

The  Acquisition  Strategy  is  related  to  the  Project  Cycle  described  in 
chapter  three  and  is  defined  as  [Ref.  31:  p.  1-1]: 

...a  business  and  technical  management  approach  designed  to 
achieve  program  objectives  within  specified  resource  constraints.  It  is  the 
framework  for  planning,  organizing,  staffing,  coordinating,  and  leading  a 
program.  It  provides  a  master  schedule  for  research,  development,  test, 
production,  fielding,  and  other  activities  essential  for  program  success,  and 
for  formulating  functional  strategies  and  plans. 

This  topic  will  use  a  Major  Defense  Acquisition  Program  for  the  focus  of 


its  content. 


b)  Acquisition  Strategy  for  a  Major  Automation  Information 
System  (MAIS) 

An  Acquisition  Strategy  for  an  MAIS  contains  the  same  conceptual  bases 
for  the  overall  plan  for  program  execution  as  an  MDAP.  In  fact,  the  DSMC’s  Acquisition 
Strategy  Guide  [Ref.  31]  makes  no  distinction  between  developing  an  Acquisition 
Strategy  for  an  MDAP  or  for  an  MAIS.  However,  there  is  enough  of  a  distinction  between 
the  two  types  of  programs  [Ref.  29]  that  the  Acquisition  Strategy  for  an  MAIS  does 
warrant  a  different  emphasis  and  therefore  its  own  topic. 

c)  System  Engineering  Process 

The  System  Engineering  Process  is  used  to  [Ref.  32:  4.3]: 

. .  .transform  operational  needs  and  requirements  into  an  integrated 
system  design  solution  through  concurrent  consideration  of  all  life-cycle 
needs  (i.e.,  development,  manufacturing,  test  and  evaluation,  verification, 
deployment,  operations,  support,  training  and  disposal).  The  system 
Engineering  process  shall  establish  a  proper  balance  between  performance, 
risk,  cost,  and  schedule,  employing  a  top-down  iterative  process  of 
requirements  analysis,  functional  analysis  and  allocation,  design  synthesis 
and  verification,  and  system  analysis  and  control. 
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The  System  Engineering  Process  is  closely  related  to  the  Technical  Aspect 
of  the  Project  Cycle.  In  fact,  Forsberg,  Mooz,  and  Cotterman  define  the  System 
Engineering  Process  as  the  application  of  the  two  subprocesses  within  their  Vee-model: 
the  System  Analysis  and  Design  Process  and  the  System  Integration  and  Verification 
Process.  [Ref.  22:  p.  37] 

d)  Defining  Critical  Design  &  Capabilities 

This  topic  encompasses  the  objectives  of  the  original  SAFE  program  and 
could  easily  be  developed  into  three  or  four  sequential  modules.  The  overall  intent  is  to 
allow  students  to  exercise  their  analytic  abilities  and  knowledge  of  systems  acquisition. 
The  module(s)  should  allow  students  to  make  trade-off  decisions  concerning  the  cost, 
schedule  and  performance  aspects  at  key  decision  points.  The  listing  of  specific  tasks 
include  [Ref.  33]: 

-Conduct  requirements  and  trade-off  analysis. 

-Identify,  analyze,  and  assess  program  risks. 

-Evaluate  contractor  proposals. 

-Develop  cost  estimates,  budget  submissions,  acquisition 

strategies,  and  program  baselines. 

-Identify  critical  acquisition  management  issues  and  prepare 

recommendations  for  decision-makers. 

This  topic  is  a  compilation  of  several  of  the  Project  Management  Elements 
and  the  Project  Cycle's  Budgeting,  Business,  and  Technical  aspects. 

e)  Risk  Management 

DoD  5000.2-R  states  that  Program  Managers  need  to  institute  a  risk 
management  program  for  each  acquisition  program  “to  identify  and  control  performance, 
cost,  and  schedule  risks.”  [Ref.  32:  3.3.2]  Risk  Management  in  the  DoD  parallels  the 
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Risks  and  Opportunities  element  of  the  Project  Management  Elements  and  is  defined  as 
“All  plans  and  actions  taken  to  identify,  assess,  mitigate,  and  continuously  track,  control, 
and  document  program  risks.”  [Ref.  3] 

j)  Analysis  &  Control 

While  Risk  Management  is  the  process  to  identify,  assess,  track,  and 
mitigate  critical  items,  Analysis  and  Control  activities  serve  as  a  basis  for  measuring 
progress  [Ref.  34:  p.  5-3]: 

DoD  5000.2R,  Part  4  states  that  performance  metrics  must  be 
developed  and  maintained  throughout  the  process  (SE  process).  These 
metrics  will  be  used  to  assess  how  well  the  evolving  design  meets  the 
system  requirements... Particular  attention  will  be  paid  to  those  metrics  that 
are  critical  to  risk  management... The  data  for  each  metric  will  be  based  on 
engineering  judgment,  design  analysis,  test  data  (including  early  test 
results),  and  operational  data,  depending  on  the  status  of  the  design. 

Analysis  and  Control  is  related  to  three  of  the  Project  Management 
Elements:  Project  Control,  Project  Visibility,  and  Project  Status. 

g)  Acquisition  Logistics 

Acquisition  Logistics  is  a  topic  so  broad  that  it  could  easily  provide 
material  for  several  modules.  It  is  broad  because  it  is  a  multi-functional  technical 
management  discipline  that  encompasses  design,  development,  test,  production,  fielding, 
sustainment,  and  improvement  modifications  that  achieve  the  user’s  peacetime  and 
wartime  readiness  requirements  [Ref.  35:  4-1].  DoD  Regulation  5000.2-R  mandates  that 
[Ref.  32:  4.3.3]: 

The  PM  shall  conduct  acquisition  logistics  management  activities 
throughout  the  system  development  to  ensure  the  design  and  acquisition  of 
systems  that  can  be  cost-effectively  supported  and  to  ensure  that  these 
systems  are  provided  to  the  user  with  the  necessary  support  infrastructure 
for  achieving  the  user’s  peacetime  and  wartime  readiness  requirements. 
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Because  Acquisition  Logistics  is  so  broad  in  scope,  it  can  be  related  to 
several  of  the  concepts  and  elements  in  the  Project  cycle  and  the  Project  Management 
Elements  as  it  is  applied  throughout  system  development  as  mandated  by  the  DoD. 

h)  Contracting  &  Contract  Management 

There  are  a  myriad  of  contract  types.  The  specific  type  is  selected  to 
provide  for  a  reasonable  sharing  of  risk  for  the  performance  of  the  contract.  Because  of  its 
ability  to  balance  the  risk,  the  selection  of  the  most  appropriate  type  of  contract  and  fee 
for  the  particular  procurement  is  essential.  [Ref.  36:  p.  1-1] 

This  topic  corresponds  to  the  Source  Selection  Phase  of  the 
Implementation  Period  from  the  Project  Cycle. 

i)  Budgeting 

Budgeting  involves  the  short  and  long  range  planning  and  execution  of 
resources.  The  improper  application  of  this  topic  means  that  there  may  be  no  program 
[Ref.  32:  2.5.2]: 

No  acquisition  program  shall  be  approved  to  proceed  beyond 
program  initiation  unless  sufficient  resources,  including  manpower,  are 
programmed  in  the  most  recently  approved  FYDP,  or  will  be  programmed 
in  the  next  Program  Objective  Memorandum  (POM),  Budget  Estimate 
Submission  (BES),  or  President's  Budget  (PB). 

Budgeting  in  acquisition  relates  to  the  budgeting  aspect  of  the  Project 


Cycle. 
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j)  Test  &  Evaluation 

Test  and  evaluation  (T&E)  in  a  defense  system’s  development  and 
acquisition  program  helps  to  identify  the  areas  of  risk  to  be  reduced  or  eliminated  [Ref. 
37:  p.  1-1]: 

Test  and  evaluation  programs  shall  be  structured  to  provide 
essential  information  to  decision-makers,  verify  attainment  of  technical 
performance  parameters,  and  to  determine  whether  systems  are 
operationally  effective,  suitable,  and  survivable  for  intended  use. 

This  topic  corresponds  to  the  Verification  Phase  of  the  Implementation 
Period  from  the  Project  Cycle. 

k)  Production  &  Fielding 

During  this  phase  of  the  DoD  acquisition  process,  the  Program  Manager 
must  ensure  the  system  meets  the  user’s  needs  while  being  sensitive  to  the  impact  of  the 
manufacturing  decisions  on  production  costs.  Through  careful  selection  from  the 
manufacturing  alternatives,  the  PM  can  achieve  significant  cost  savings.  The  key 
activities  for  this  phase  include  manufacturing,  contract  monitoring,  and  acceptance 
testing.  [Ref.  38:  p.  3-11] 

This  topic  corresponds  to  the  Production  Phase  of  the  Project  Cycle. 

l)  Work  Breakdown  Structure  &  Earned  Value 

The  DoD  regulation  5000.2-R  requires  Program  Managers  to  establish  a 
program  work  breakdown  structure  (WBS)  to  provide  a  framework  for  program  and 
technical  planning,  cost  estimating,  resource  allocations,  performance  measurements,  and 
status  reporting  [Ref.  32:  4.4.2]. 
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The  WBS  corresponds  to  the  Project  Planning  element  and  EV 
corresponds  to  the  Project  Status  element  of  the  Project  Management  Elements. 

m)  Software  Management 

Software  Managers  need  to  be  able  to  “define  objectives,  to  create, 
evaluate,  and  select  alternatives  for  reaching  them,  and  to  control  the  implementation  of 
selected  alternatives”.  [Ref.  39]  These  can  be  difficult  tasks  because  of  the  intangible 
nature  of  software.  Software  estimation  is  a  key  activity  in  planning  and  managing 
software  intensive  programs.  Software  cost  and  schedule  estimation  is  usually 
implemented  using  models  that  quantitatively  relate  measures  or  projections  of  software 
size,  development  and  maintenance  effort,  schedule,  and  quality  through  the  use  of  an 
integrated  performance  or  efficiency  measure.  [Ref.  40] 

Software  Management  is  not  limited  to  any  single  portion  of  the  project 
management  principles  from  chapter  three;  instead,  one  would  use  all  the  principles  while 
emphasizing  the  software  relevant  elements  like  Project  Control,  to  effectively  manage  a 
software  project. 

n)  People  Factors 

This  item  was  not  offered  in  any  of  the  preexisting  courses.  However, 
through  the  research  and  interviews  it  surfaced  as  a  necessary  topic.  People  skills  are 
required  for  a  manager  when  building  or  interacting  with  the  project  team  and  for 
communicating  with  the  stakeholders  for  a  project.  This  topic  is  related  to  the  Teamwork 
and  Leadership  elements. 


66 


2.  Strategies 

Because  of  the  varied  student  backgrounds,  class  objectives,  and  instructor 
preferences,  there  emerged  no  single  strategy  appropriate  for  presenting  the  modules. 
Fortunately,  a  flexible,  modular  architecture  as  described  in  a  previous  section  will  allow 
an  instructor  to  use  whatever  strategy  he  or  she  determines  is  the  most  appropriate  to 
reach  the  course  obj  ectives . 

The  research  indicated  that  there  were  several  strategies  to  presenting  the  topics 
that  seemed  to  be  viable  options  for  meeting  course  objectives. 

a)  Chronological  by  Phase 

Chronological  by  phase  is  a  common  approach  to  teaching  Program 
Management.  Most  of  the  consortium  schools  teaching  Acquisition  Program  Management 
used  this  method  in  at  least  one  of  their  courses.  This  approach  takes  the  student  through 
the  acquisition  phases  in  a  sequential  manner,  introducing  each  topic  as  it  appears  in 
chronological  order.  For  example,  the  first  topic  may  be  a  Requirements  Generation 
related  topic  followed  by  a  Concept  Exploration  related  topic. 

b)  Concept  Centric 

A  concept  centric  strategy  places  the  importance  of  the  topic  or  concept 
over  the  need  for  a  chronological  ordering  of  their  appearance.  Using  this  strategy,  the 
topics  are  more  likely  to  appear  as  stand  alone  material  and  not  lead  into  the  next  module. 
For  example,  a  topic  concerning  logistics  may  follow  a  topic  involving  test  and 
evaluation. 
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c)  Parallel  Comparison  of  MDAP  vs.  MAIS 

The  comparison  strategy  emphasizes  both  types  of  program  through  out 
the  acquisition  cycle  to  provide  an  appreciation  for  their  differences.  Modules  designed 
for  this  strategy  would  probably  be  sequential  in  nature  and  the  objectives  would  focus  on 
highlighting  the  differences  between  the  management  of  these  programs. 

d)  Systems  Engineering  Centric 

A  systems  engineering  centric  approach  provides  students  a  view  of  the 
acquisition  process  through  modules  that  underscore  the  role  of  the  systems  engineering 
process. 

e)  Logistics  Centric 

A  logistics  centric  approach  uses  the  same  concept  as  the  systems 
engineering  centric  strategy  except  it  features  the  acquisition  logistic  perspective  to  the 
acquisition  systems  management  process. 

f)  Hybrid  Approach 

A  hybrid  approach  mixes  elements  from  the  preceding  strategies.  This 
style  may  involve  any  number  of  the  aforementioned  approaches  combined  to  present  an 
optimized  collection  of  strategies  for  one  course.  An  example  of  this  strategy  would  be  to 
use  a  parallel  comparison  for  the  first  two  modules,  a  chronological  by  phase  for  the  next 
few  modules,  and  then  a  few  concept  centric  for  the  final  modules. 

D.  CHAPTER  SUMMARY 

Because  it  is  impractical  for  a  single  static  program  to  address  all  the  strategies 
and  topics  required  by  the  users,  the  NPS  ATAPMS  needs  to  accommodate  many 
interchangeable  modules  and  must  scale  to  the  user’s  needs.  Additionally,  to  be  viable  for 
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the  future  and  to  comply  with  the  intent  of  the  Defense  Reform  Initiative,  the  program 
must  also  incorporate  the  Advanced  Distance  Learning  precepts.  Table  4-1  provides  a 
comprehensive  summary  of  the  recommendations  for  the  structure,  strategies,  and  topics 
for  the  application  of  an  ATAPMS  at  the  NPS  presented  in  this  chapter. 

This  chapter  used  the  Vee-model  concept  to  develop  baseline  recommendations 
for  the  structure,  and  to  propose  content  for  off-core  study.  The  next  chapter  will  continue 
this  method  by  using  a  subset  of  the  proposed  content  to  develop  a  prototype  of  an 
ATAPMS  for  the  NPS. 


Structure 

Content 

-Use  a  capstone  style  approach 
to  the  content 
-Accommodate  at  least  10 
weekly  topic  modules 
-Be  capable  of  presenting  well 
developed  thought- 
provoking  scenarios  and 
questions 

-Ensure  modular  architecture 
for  scalability  and  a  plug  and 
play  capability 
-Be  IMS  compliant 
-Ensure  Accessibility 
-Ensure  Durability 
-Ensure  Re-usability 
-Ensure  Cost  effectiveness 

-Acquisition  Strategy  for  a  Major  Defense 
Acquisition  Program  (MDAP) 

-Acquisition  Strategy  for  a  Major 

Automation  Information  System  (MAIS) 
-System  Engineering  Process 
-Defining  Critical  Design  &  Capabilities 
-Analysis  &  Control 
-Risk  Management 
-Acquisition  Logistics 
-Contracting  &  Contract  Management 
-Budgeting 
-Test  &  Evaluation 
-Production  &  Fielding 
-Work  Breakdown  Structure  &  Earned  Value 
-Software  Management 
-People  Factors 

-Chronological  by  Phase 
-Concept  Centric 
-Systems  Engineering 
Centric 

-Logistics  Centric 
-Hybrid  Approach 

Table  4-1.  Summary  Table. 
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V.  DESIGN  OF  THE  PROGRAM  MANAGEMENT  TRAINING  TOOL 


A.  INTRODUCTION 

This  chapter  continues  the  Vee-model  off-core  study  concept  by  exploring  one 
possible  approach  to  devising  an  ATAPMS  application  that  meets  the  requirements 
specified  in  Chapter  Four.  Specifically,  this  chapter  enumerates  strategies  for  acquiring 
the  software  required  to  support  an  ATAPMS  and  then  uses  one  of  those  strategies  for 
demonstrating  the  construction  of  a  notional  module. 

B.  DEVELOPMENT  STRATEGIES 

There  are  two  realistic  approaches  to  developing  an  ATAPMS  program  that  meet 
the  top-level  specifications  defined  in  the  previous  chapter:  develop  new  software  code  or 
use  commercial  based  authoring  tools. 

A  third  option,  modifying  existing  software  by  combining  applications  such  as 
Microsoft  Project  Manager  and  Microsoft  Access  using  a  web  based  interface  like  Cold 
Fusion,  is  not  reasonably  feasible.  Adding  additional  modules  in  a  "plug  and  play" 
fashion  that  incorporates  the  IMS  requirements  to  applications  cobbled  together  is  as 
involved  as  writing  new  software  code.  Further,  software  licensing  management, 
seemingly  random  changing  of  application  versions,  and  lack  of  vendor  support  because 
of  modifications  made  to  their  products,  increases  the  undesirability  of  this  option. 

The  decision  between  the  two  remaining  options  for  the  best  approach  depends  on 
several  factors.  In  the  case  of  the  NPS,  the  most  applicable  factors  according  to  the 
interviews  were: 

-Ease  of  use:  The  ability  for  an  instructor  to  use  the  software  to  create  or 
modify  modules. 
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-Applicability:  The  capability  of  the  program  to  help  the  instructor  meet 
his  or  her  learning  objectives. 

-Cost:  The  amount  of  resources  measured  in  time  and  money  required  to 
create  modules. 

-Maintainability:  The  ability  to  modify,  correct  faults,  improve 
performance,  or  other  attributes,  or  adapt  to  a  changed  environment. 

-Time:  The  amount  of  time  required  to  field  the  initial  and  subsequent 
modules. 

1.  Developing  New  Code 

The  first  method  is  developing  the  application  using  completely  new  software 
code.  This  method  has  one  strong  benefit.  An  application  designed  from  the  beginning 
with  the  specifications  and  requirements  of  the  NPS  would  optimize  the  “applicability” 
criterion,  and  as  a  result  be  highly  tailored  to  the  needs  of  the  acquisition  program 
management  instructors. 

According  to  an  NPS  software  instructor  interviewed,  however,  the  disadvantages 
to  building  an  application  to  meet  the  top-level  specifications  identified  for  an  ATAPMS 
by  writing  new  code  are  numerous.  Time  required  for  development  of  the  initial  and 
subsequent  modules  is  a  significant  factor  for  this  option.  Also,  to  obtain  the  ease  of  use 
and  applicability  desired,  the  cost  for  building  a  proprietary  program  will  be  much  higher 
than  purchasing  a  commercial  product. 

Maintenance  requirements  add  another  heavy  cost  burden  to  the  builder  as  well. 
For  example,  if  the  ATAPMS  were  programmed  in  JAVA,  a  language  that  adheres  to  the 
top-level  specifications  enumerated  in  the  previous  chapter,  all  content  authors  needing  to 
create  or  modify  existing  or  subsequent  modules  must  adhere  to  the  same  standards  used 
to  create  the  original  program.  While  still  feasible,  this  restriction  makes  the  concept  of  a 
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“plug  and  play”  program  harder  to  accomplish,  especially  as  institutional  knowledge  of 
the  standards  for  the  original  program  declines  over  time. 

Additionally,  the  ability  for  the  average  instructor  to  create  and  employ  a  module 
becomes  highly  unlikely  in  this  scenario.  Because  of  the  requirement  to  write  code 
specific  to  the  original  program,  modifications  to  existing  modules  or  the  creation  of 
additional  modules  will  probably  require  the  efforts  of  an  experienced  programmer. 

Finally,  the  adherence  of  the  software  to  the  IMS  standards  adds  a  complex  and 
costly  dimension  to  the  generation  of  the  software  code.  Not  only  will  the  modules 
require  a  programmer’s  skill  to  write,  but  that  programmer  must  also  be  familiar  with  the 
nuances  of  the  IMS  standards. 

2.  Commercial  Based  Software 

The  second  option  for  developing  the  ATAPMS  application  is  to  use 
commercially  available  authoring  products.  Authoring  systems  are  software  applications 
that  allow  the  developer  to  create  a  multimedia  education  product  without  having  to  write 
software  code.  Each  authoring  system  has  its  strengths:  some  work  best  with  databases, 
computer-based  instruction,  or  just  as  general  all-purpose  applications.  [Ref.  41] 

The  commercial  authoring  systems  have  one  major  disadvantage,  but  they  also 
have  several  distinct  advantages.  The  disadvantage  to  using  a  commercial  product  is  the 
“applicability”  factor.  Because  the  functionality  of  a  commercial  product  is 
predetermined,  there  may  be  features  that  an  instructor  needs  that  are  not  available.  This 
disadvantage  caused  Indiana  University  to  choose  creating  its  online  course  management 
system  by  writing  new  code  instead  of  adapting  a  commercially  available  application 
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[Ref.  42].  Although  the  Indiana  University’s  requirements  are  different  from  the  NPS 
requirements,  this  example  illustrates  the  importance  of  the  “applicability”  issue. 

The  advantages  to  using  commercial  authoring  tools  that  allow  one  to  create 
custom  courseware  are  numerous.  The  first  advantages  are  cost  and  time.  The  average 
cost  for  a  commercial  authoring  system  typically  range  from  $1,000.00  to  $5,000.00  [Ref. 
41],  which  according  to  one  NPS  software  instructor  is  several  times  less  than  writing 
new  software  code  for  a  project  like  an  ATAPMS.  That  cost  differentiation  widens 
considerably  with  the  need  for  new  modules  and  support  for  the  existing  software.  Also, 
the  time  needed  to  develop  the  initial  application  or  subsequent  modules  is  measured  in 
days  or  hours,  not  months.  This  advantage  is  especially  significant  for  instructors  who 
need  to  make  last  minute  modifications  to  their  courses. 

The  third  advantage  is  maintainability.  The  commercial  vendors  have  an  inherent 
need  to  keep  pace  with  the  evolving  technologies  in  a  cost  effective  and  timely  manner. 
This  condition  works  to  the  user's  favor  because  it  shifts  the  burden  for  maintaining  the 
software  code  from  the  user  to  the  vendor.  Because  the  vendor  has  a  comparative 
advantage  in  maintaining  the  software,  it  will  cost  less  for  the  user  to  adapt  the  existing 
software  to  a  changed  environment. 

A  fourth  advantage  is  the  ability  for  the  average  instructor  to  generate  his  or  her 
own  modules  without  the  need  for  programming  expertise.  The  web  based  authoring  tools 
that  meet  the  ATAPMS  top-level  specifications  use  a  graphical  user  interface  that  allow 
the  user  to  create  a  multimedia  product  without  having  to  write  software  code. 

A  final  advantage  is  the  capability  for  the  NPS  to  eventually  integrate  the  use  of 
an  on-line  management  software  system  to  organize  the  delivery  of  the  ATAPMS 
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courseware.  While  this  criterion  was  not  mentioned  by  any  of  the  instructors  interviewed, 
the  research  indicates  that  given  sufficient  growth  in  the  use  of  an  ATAPMS,  an  on-line 
management  tool  may  be  essential  to  successfully  scaling  the  ATAPMS  to  a  wider 
audience.  If  the  use  of  the  ATAPMS  grows  from  a  once  a  week  resident  laboratory 
exercise  to  a  distance  learning  course  supplement,  instructors  may  need  such  a  tool  to 
help  administer  the  ATAPMS  and  other  related  courseware.  The  management  system 
would  assist  in  administering  the  ATAPMS  by  providing  instructors  the  ability  to 
structure  learning  groups,  add  users,  control  the  course  availability,  and  obtain  detailed 
evaluation  reports  on  student  performance.  [Ref.  43] 


The  research  demonstrates  that  to  meet  the  initial  needs  of  the  NPS,  investing  in  a 
commercial  authoring  tool  to  develop  an  ATAPMS  has  an  advantage  in  many  areas  over 
designing  a  proprietary  application  (Table  5-1). 


Ease  of 
Use 

Applicability 

Time 

Cost 

Maintainability 

Management 

System 

Commercial 

Authoring 

Tool 

4- 

- 

+ 

+ 

+ 

+ 

Proprietary 

Application 

+ 

+ 

— 

— 

- 

Table  5-1.  Option  Comparison. 


C.  CONSTRUCTING  THE  MODULES 

This  section  uses  a  subset  of  the  proposed  content  from  the  previous  chapter  and  a 
commercial  authoring  tool  to  illustrate  one  way  of  developing  an  ATAPMS.  The 
following  off-core  study  uses  the  “hybrid”  approach,  mixing  the  elements  from  the 
preceding  types  of  strategies  documented  in  Chapter  Four  to  present  the  topic  modules.  In 
the  planning  phase,  the  titles  of  10  modules  are  listed  to  illustrate  a  notional  course  load 
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for  a  quarter.  From  these  10  topics,  four  modules  are  presented  in  a  high  level  format  to 
demonstrate  their  learning  points.  In  the  layout  phase,  one  of  these  modules  is  further 
developed  using  a  commercial  authoring  application. 

1.  Planning 


Table  5-2  depicts  the  10  modules  selected  from  the  topic  section  of  the  previous 
chapter. 


Week  1 

Week  2 

Week  3 

Week  4 

Week  5 

Acquisition 

Strategy  (MDAP) 

Acquisition 

Strategy  (MAIS) 

System 

Engineering 

Process 

Define  Critical 
Design  & 
Capabilities  I 

Define  Critical 
Design  & 
Capabilities  II 

Week  6 

Week  7 

Week  8 

Week  9 

Week  10 

Define  Critical 
Design  & 
Capabilities  III 

Risk  Management 

Analysis  &  Control 

Acquisition 

Logistics 

Contracting  & 

Contract 

Management 

Table  5-2.  Notional  Course  Load. 


To  continue  the  planning  process,  the  learning  objectives  for  modules  1,2,7  and  8  - 
are  described  using  the  U.S.  Army’s  “Task,  Condition,  Standards”  method  (Table  5-3), 
followed  by  a  "design  approach"  (Table  5-4)  description  for  constructing  the  modules. 
The  task  defines  a  measurable  activity  for  the  students  to  perform  [Ref.  44].  The 
conditions  explain  the  circumstances  and  environment  in  which  the  task  is  to  be 
performed.  The  standard  describes  the  minimum  acceptable  proficiency  required  in  the 
performance  of  the  particular  training  task.  Finally,  the  design  approach  description 
provides  general  guidance  for  features  and  functionality  required  from  the  software  to 
allow  the  module  to  accomplish  the  intended  learning  objectives. 
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Module  1  Module  2  Module  7  Module  8 

Acquisition  Strategy  Acquisition  Risk  Management  Analysis  and  Control 

(MDAP) _ Strategy  (MAIS) _ 

TASK:  TASK:  TASK:  TASK: 

Develop  an  Develop  an  Evaluate  a  program  Employ  metrics 

Acquisition  strategy  for  a  Acquisition  strategy  for  scenario  and  determine  developed  during  Risk 

Major  Defense  a  Major  Acquisition  the  areas  of  risk.  Management  Module 

Acquisition  Program.  Information  System.  prioritize  the  selected  and  interpret  results  from 

areas  and  devise  a  risk  a  visual  display  of  the 
mitigation  strategy  that  submitted  metrics. 

_ includes  metrics. _ 

CONDITION:  CONDITION:  CONDITION:  CONDITION: 

Given  a  Mission  Needs  Given  a  Mission  Given  a  program  Given  a  program 

Statement,  an  Needs  Statement,  an  scenario  with  sufficient  scenario  that  provides 

Acquisition  Decision  Acquisition  Decision  cost  and  probability  of  quantifiable  data 

Memorandum,  Progress  Memorandum,  Progress  occurrence  details,  risk  reflecting  a  snapshot  in 

Report  (CE  Phase  Report  (CE  Phase  reduction  strategies  and  time  for  a  project  status. 

Studies),  Performance  Studies),  Performance  a  risk  reduction  budget. 

Requirements,  and  a  Requirements,  and  a 

general  scenario. _ general  scenario. _ 

STANDARD:  STANDARD:  STANDARD:  STANDARD: 

Student  teams  must  Student  teams  must  Student  teams  must  Student  teams  must 

determine  an  appropriate  determine  an  demonstrate  a  working  analyze  given  program 

Acquisition  strategy  for  appropriate  Acquisition  level  of  knowledge  of  data  and  extract  the 

an  MDAP.  They  must  strategy  for  an  MAIS.  Risk  Management.  appropriate  information 

adequately  address  the  They  must  adequately  Teams  should  be  able  to  needed  for  input  to  a 

following  elements  in  address  the  following  analyze  a  scenario  and  mechanism  that  will 

their  strategy:  elements  in  their  identify  the  areas  of  risk  provide  a  visual  read-out 

-basic  strategy  (in  terms  strategy:  and  produce  a  Risk  of  the  submitted  metrics, 

of  milestones,  phases,  -basic  strategy  (in  terms  Management  strategy  Students  must  evaluate 

technical  reviews,  of  milestones,  phases,  (to  include  a  the  visual  output  and 

contract  award  dates  and  technical  reviews,  quantifiable  measure  of  provide  mid-level 

other  key  events)  contract  award  dates  risk  for  each  event).  management  analysis  and 

-Streamlining,  and  other  key  events)  Teams  will  perform  a  recommendations  to 

-Competition  -Streamlining,  cost/benefit  analysis.  senior  level  management. 

-Contract  types  -Competition  Teams  will  also  identify 

-Proposed  exit  criteria  -Contract  types  the  metrics  to  be 

for  the  next  phase  -Proposed  exit  criteria  included  as  reportable 

-Budgeting  for  the  next  phase  items  in  the  contract. 

_ -Budgeting _ 

Table  5-3.  Sample  Learning  Objectives. 

This  design  approach  uses  generally  accepted  design  principles  that  are  already 
included  as  a  feature  in  several  authoring  tools.  For  example,  the  following  design 
approach  includes  the  need  for  interactivity  and  user  friendly  navigation. 
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Module  1 

Acquisition  Strategy 
(MDAP) 

Module  2 

Acquisition  Strategy 
(MAIS) 

Module  7 

Risk  Management 

Module  8 
Analysis  and 
Control 

Design  Approach: 

(a)  Make  hyperlinks 
available  on  the  first 
page  along  with 
instructions  for 
completing  the  module. 
Links  correspond  to  the 
read-ahead  (documents 
describing  the  scenario 
for  this  module,  and  the 
instructions  for  fulfilling 
this  module's 
requirements)  and  the 
Defense  Acquisition 
DeskBook  Online. 

(b)  There  are  six  pages 
with  blank  window 
backgrounds,  each 
displaying  24  evenly 
spaced  vertical  lines 
delineating  the  months. 

(c)  Movable  bar  and 
triangle  shaped  icons 
representing  the  various 
tasks  possible  to  perform 
dining  that  phase  (e.g. 
conduct  Analysis  of 
Alternatives,  etc.)  are 
available  at  the  bottom  of 
the  window  for  students 
to  "drag  and  drop"  in  the 
position  predetermined 
by  their  review  of  the 
scenario. 

Design  Approach: 

(a)  Make  hyperlinks 
available  on  the  first  page 
along  with  instructions  for 
completing  the  module. 
Links  correspond  to  the 
read-ahead  (documents 
describing  the  scenario  for 
this  module,  and  the 
instructions  for  fulfilling 
this  module’s 
requirements)  and  the 
Defense  Acquisition 
DeskBook  Online. 

(b)  There  are  six  pages 
with  blank  window 
backgrounds,  each 
displaying  24  evenly 
spaced  vertical  lines 
delineating  the  months. 

(c)  Movable  bar  and 
triangle  shaped  icons 
representing  the  various 
tasks  possible  to  perform 
during  that  phase  (e.g. 
conduct  Analysis  of 
Alternatives,  etc.)  are 
available  at  the  bottom  of 
the  window  for  students  to 
"drag  and  drop"  in  the 
position  predetermined  by 
their  review  of  the 
scenario. 

Design  Approach: 

(a)  Make  hyperlinks 
available  on  the  first 
page  along  with 
instructions  for 
completing  the  module. 
Links  correspond  to  the 
read-ahead  (documents 
describing  the  scenario 
for  this  module,  and  the 
instructions  for  fulfilling 
this  module's 
requirements)  and  the 
Defense  Acquisition 
DeskBook  Online. 

(b)  The  next  page 
provides  a  table  with  text 
and  number  capture 
windows  that  students 
fill  in  with  event  name, 
probability,  and  impact 
of  occurrence. 

(c)  The  next  page(s) 
presents  the  user  with  a 
calculated  risk  exposure 

;  figure  and  requires 
!  students  to  prioritize  risk 
'  reduction  measures 
based  on  a  cost  benefit 
analysis. 

Design  Approach: 

(a)  Make  hyperlinks 
available  on  die  first 
page  along  with 
instructions  for 
completing  the  module. 
Links  correspond  to  the 
read-ahead  (documents 
describing  the  scenario 
for  this  module,  and  the 
instructions  for 
fulfilling  this  module's 
requirements)  and  the 
Defense  Acquisition 
DeskBook  Online. 

(b)  The  next  several 
pages  allow  students  to 
input  the  raw  data 
gleaned  from  the 
scenario  for  the  relevant 
metrics  identified 
during  Module  7. 

(c)  The  next  page 
provides  a  display  using 
graphs,  charts  or  other 
appropriate  visual 

I  display  of  the  data 
entered. 

Table  5-4.  Design  Approach. 
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Module  1 
Acquisition 
Strategy  (MDAP) 

Module  2 

Acquisition  Strategy 
(MAIS) 

Module  7 

Risk  Management 

Module  8 

Analysis  and  Control 

(d)  The  next  page 
prompts  the  user  to 
enter  a  written 
summary  that 
characterizes  their 
overall  acquisition 
strategy. 

(e)  Follow  on  pages 
require  the  students  to 
answer  questions  using 
text  capture  boxes 
addressing  their 
approach  to  selecting 
contract  types,  CAIV 
considerations, 
budgeting  breakdown, 
and  IPT 

implementation  as  they 
relate  to  an  MDAP 
program. 

(d)  The  next  page 
prompts  the  user  to  enter 
a  written  summary  that 
characterizes  their 
overall  acquisition 
strategy  and  how  that 
strategy  differs  from  an 
MDAP  strategy. 

(e)  Follow  on  pages 
require  the  students  to 
answer  questions  using 
text  capture  boxes 
addressing  their 
approach  to  selecting 
contract  types,  CAIV 
considerations, 
budgeting  breakdown, 
and  IPT  implementation 
as  they  relate  to  an 

MAIS  program. 

(d)  Follow-on 
page(s)  query 
students  for  metrics 
to  include  as 
reportable  items  in 
the  contract  to 
monitor  risk  factors. 

(e)  Follow  on  pages 
require  the  students 
to  answer  questions 
using  text  capture 
boxes  addressing 
their  overall  risk 
Management 

Strategy. 

(d)  The  next  page(s) 
provides  boxes  arranged 
in  an  ordinal  sequence 
to  allow  students  to 
express  their  qualitative 
opinion  about  the  status 
of  a  program  via 
checkmarks. 

(e)  Follow  on  page(s) 
use  text  capture  boxes 
requiring  the  students  to 
provide  assessment  of 
overall  program  status 
as  they  interpret  it  from 
the  displayed  data  and 
to  make  any 
recommendation  for 
senior  management 
concerning  necessary 
corrective  actions. 

Table  5-4.  Design  Approach  (continued). 


2.  Layout 

Having  demonstrated  a  method  of  conducting  the  planning  phase,  the  next  step  is 
to  use  one  of  the  preceding  module  plans  and  develop  an  ATAPMS  module  using  a 
commercial  authoring  tool.  This  section  illustrates  how  an  ATAPMS  module  would  look 
from  the  developer  and  the  user  perspective. 

The  authoring  tool  used  to  demonstrate  this  off-core  study  is  "ToolBook  II, 
Assistant  7"  distributed  by  Asymetrix  Learning  Systems.  While  there  are  commercially 
available  authoring  tools,  such  as  MacroMedia's  "Authorware",  that  contain  the 
functionality  required  for  an  ATAPMS,  ToolBook  13  is  not  among  them.  ToolBook  II 
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does  not  possess  the  text  capture  and  retrieval  functionality  necessary  to  facilitate  a 
qualitative  review  of  the  user's  input.  Therefore,  ToolBook  II  is  used  in  this  section  only 
to  generate  the  figures  needed  to  illustrate  the  user's  view  of  a  module  that  appears  to 
contain  the  functionality  desirable  for  an  ATAPMS. 

a)  Module  Posting 

Once  the  module  is  created,  ToolBook  II  provides  the  option  to  "Import  to 
HTML":  a  feature  that  automatically  packages  the  module  in  a  directory  ready  for 
publication  on  a  website  (Figure  5-1).  Within  each  directory  are  all  of  the  files  necessary 
for  posting  the  module  on  a  web  server  (Figure  5-2).  Because  all  of  these  files  are 
automatically  generated  by  the  authoring  tool  with  the  execution  of  one  command,  an 
instructor  acting  as  developer  only  needs  to  be  proficient  with  developing  the  actual 
content  of  the  module,  and  not  the  secondary  task  of  preparing  the  module  to  be  accessed 
from  a  web  server. 
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1  Contents  of  ATAPMS  Modules* 


I  module_1 
I  module_2 
I  module_3 
I  module_4 
!  module_5 
I  module_6 
I  module  7 


I  module_9 
I  module  1 0 


Figure  5-1.  Web  Enabled  Directories. 


1  Contents  of  ,module_84 


bootstrap 

bootstrap 

index 

readme 

scriptlet 

unsupported 


jlOobject(s) 


1 34.9KB  (Disk  free  space:  1 .26GB) 

Figure  5-2.  Inside  Module  8’s  Directory. 
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b)  Module  Presentation 

After  loading  the  module  directories  onto  a  web  server,  a  simple  web  page 
is  all  that  is  needed  to  provide  a  link  to  each  of  the  scenarios  and  module  directories.  The 
instructor  can  use  this  web  site  as  a  "home  page"  for  the  courseware,  allowing  students 
convenient  access  to  the  scenarios  and  modules  (Figure  5-3).  Students  can  access  this  site 
from  any  location  on  the  Internet  well  in  advance  in  order  to  prepare  for  the  scheduled 
lab. 

This  schema  supports  the  plug  and  play  concept  by  using  an  easily 
reconfigurable  interface  between  the  application  modules  and  the  students.  Any  changes 
made  by  the  instructor  or  application  manager  to  the  modules,  their  directory  structures, 
or  the  scenarios,  are  transparent  to  the  users.  Any  additions  made  to  the  courseware  can 
be  easily  and  quickly  accommodated  by  the  web  interface. 

c)  Module  Example 

The  following  module  example  demonstrates  one  method  of  designing  a 
desirable  learning  experience  for  a  user  based  on  the  preceding  "Sample  Learning 
Objectives"  and  "Design  Approach"  presented  in  Table  5-3  and  5-4  for  "Module  8: 
Analysis  and  Control".  Prior  to  downloading  the  module,  the  user  would  first  download 
and  review  a  case  scenario  (e.g.  "Scenario  8"  shown  in  Figure  5-3)  that  is  also  linked  to 
course  home  page.  Once  familiar  with  the  scenario,  the  user  can  return  to  the  home  page 
and  download  the  actual  module  (e.g.  "Lab  8  Module"  shown  in  Figure  5-3). 
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File  £dt  View  Go  Communicator  Help 

1  £  '  &  3  5  S- 

|  Back  Forward  Reload  Home  Search  Netscape  Print  Security 
:|  Bookmarks  Jfe.  Go  to: [http: //sm/nps. navy.mil/curric/MN4444/modules.htrnl 


;.~v  Jg 

r  What's  Related 


Welcome  to  MN  4444  "Acquisition  Program  Management" 


!aS*  |H]>= 


Module 

1  :  Acquisition  Strategy  (MDAP) 

Scenario  1 

Lab  1  Module 

Module 

2  :  Acquisition  Strategy  (MATS) 

Scenario  2 

Lab  2  Module 

Module 

3  :  Systems  Engineering  Process 

Scenario  3 

Lab  3  Module 

Module 

4  :  Eiefine  Critic  el  Design  &  Capabilities  I 

Scenario  4 

Lab  4  Module 

Module 

5  :  Define  Critic  -el  De  sign  &  Capabilities  II 

Scenario  5 

Lab  5  Module 

Module 

6  :  Define  Critical  Design  dz  Capabilities  III 

Scenario  6 

Lab  6  Module 

Module 

7  :  Risk  Management 

Scenario  7 

Lab  7  Module 

Module 

2  :  Analysis  &  Control 

Scenario  2 

Lab  2  M  o  dule 

Module 

9  :  Acquisition  Logistics 

Scenario  9 

Lab  9  M  o  dule 

Module 

10  :  Contracting  &  Contract.  Management 

Scenario  10 

Lab  10  Module 

j  Document  Done 

=| 

Figure  5-3.  Web  Site  Used  as  a  "Home  Page." 


Figure  5-4  provides  an  example  module  front  page  as  viewed  by  the  user 
after  downloading  it  from  the  home  page.  In  this  example,  the  module  fosters  an 
integrative  capstone  approach  to  the  lesson  and  therefore  is  designed  be  used  by  student 
teams.  Accordingly,  the  users  would  enter  their  "team  identification"  so  the  instructor 
could  distinguish  between  the  various  teams'  module  outputs.  There  is  also  an 
administrative  information  and  HTTP  hyper-link  to  the  scenario  read-ahead  and  the 
Department  of  Defense  Acquisition  DeskBook  web  site. 
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I  ft  Module  8 


File  Edit  View  Go  Text  Help 


4444  Module  8 


Analysis  and  Control 


mmm 


ecome  to  Module  8  :  Analysis  and  Control 


rerequisites:  Completion  of  Module  7  "Risk  Management 


lease  Enter  Your  Team  ID: 


Link  to  Case  Study  Read-ahead 


Figure  5-4.  Module  Front  Page. 

The  next  page  provides  an  example  of  requiring  students  to  extract  the 
appropriate  information  to  demonstrate  the  understanding  of  a  scenario.  The  module  page 
presents  a  query  to  the  user.  The  previously  viewed  scenario  concerning  the  application  of 
software  metrics  provided  the  user  with  the  quantitative  software  metrics  data  required  to 
answer  the  query.  Figure  5-5  shows  the  student’s  input  to  those  queries  based  on  their 
understanding  of  the  scenario.  Although  it  is  not  shown  in  this  example,  there  would  be 
as  many  subsequent  pages  as  necessary  to  capture  the  relevant  metrics. 


Link  to  DoD  DeskBook 
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File  Edit  View  Go  Text  Help 


4444  Module  8 


Analysis  and  Control 


ATAPMS 


Based  on  your  review  of  the  scenario,  enter 
the  relevant  metrics  below: 


Figure  5-5.  Numerical  Input. 


After  requiring  the  students  to  demonstrate  an  ability  to  distill  factual 
material  from  the  given  scenario,  the  following  pages  prompt  them  to  provide  qualitative 
interpretations  of  the  factual  data.  The  next  few  pages  of  the  module  query  the  students 
for  their  assessment  of  the  program's  status  based  only  on  the  numerical  data  (Figure  5-6). 


The  students  are  prompted  to  evaluate  the  status  of  the  various  components  (e.g. 
"requirements"  or  "design")  of  the  scenario  program  based  only  on  the  reading  of  the 
scenario.  In  this  example,  the  students  will  check  the  block  to  provide  their  assessment. 


Again,  although  it  is  not  shown  in  this  example,  there  would  be  as  many  subsequent 
pages  as  necessary  to  capture  the  assessment  of  all  the  desired  areas. 
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After  the  students  complete  their  assessment,  the  follow-on  page  of  the 
module  then  provides  a  graphic  that  transforms  the  numerical  data  into  a  visual  display 
(Figure  5-7).  Next,  to  help  illustrate  the  usefulness  of  comprehensive  graphics,  the 
module  allows  the  students  to  reevaluate  the  previously  answered  questions  (Figure  5-8). 
The  intent  of  this  comparison  is  to  measure  the  difference  in  the  student's  perception  of 
the  data  based  on  numerical  versus  a  visual  reference  and  demonstrate  the  advantage  of  a 
graphical  presentation  for  managers. 
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Link  to  Case  Study  Read-ahead 


iffi  J 

nisi 

Link  to  DoD  DeskBook 

4  1 

Figure  5-6.  Query  Based  on  Numerical  Data. 
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Link  to  Case  Study  Read-ahead 


Link  to  DoD  DeskBook 


Figure  5-8.  Query  Based  on  Visual  Display. 
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The  final  pages  of  this  module  require  the  students  to  elaborate  on  their 
overall  assessment  of  the  program  status.  Figure  5-9  requires  the  students  to  type  their 
evaluation  of  the  program's  "health"  into  a  text  capture  box.  Figure  5-10  requires  the 
students  to  type  into  a  text  capture  box  their  recommendations  for  corrective  actions.  This 
feedback  allows  an  instructor  to  measure  the  student's  comprehension  of  a  program’s 
status. 
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1  Ar  Module  8 

HiiBl 

File  Edit  View  Go  lent  Help 

m  ™  S  e  e  « m  tOMBI 

ATAPMS 

Analysis  and  Control 

Based  on  your  assessment  of  the  indicators  displayed  in  the  previous  graphic, 
provide  any  recommendations  you  would  make  for  corrective  action  below: 

Link  to  Case  Study  Read-ahead 
[Link  to  DoD  DeskBook 

Figure  5-10.  Recommendation  for  Corrective  Action. 

d)  Module  Output 

The  format  of  a  module's  output  varies  with  the  different  commercial 
authoring  tools.  A  review  of  several  of  the  tools  available  shows  that  the  feedback  options 
have  a  wide  range.  Some  authoring  tools  provide  an  instructor  with  only  a  simple 
numerical  score  based  on  the  user's  performance.  Others  offer  the  instructor  the  capability 
to  view  the  user's  score  and  the  specific  answers  chosen  for  the  questions  presented.  The 
most  useful  authoring  tools,  and  the  only  category  that  meets  the  requirements  for  an 
ATAPMS,  provide  the  capability  to  export  all  the  information  entered  by  the  user. 

The  ability  to  capture  and  retrieve  all  of  the  user's  input  is  essential.  This 
feature  is  necessary  to  allow  an  instructor  to  perform  a  qualitative  review  of  essay  type 


89 


answers.  The  capture  and  retrieval  capability  is  also  necessary  for  writing  numerical  input 
to  a  database  and  then  developing  dynamic  graphics  from  that  information. 

D.  CHAPTER  SUMMARY 

The  most  practical  option  for  building  the  ATAPMS  modules  is  through  the  use 
of  commercial  based  authoring  tools.  These  tools  have  many  advantages  over  developing 
new  code:  developing  new  modules  or  adjusting  existing  modules  is  easier  and  costs  less, 
requires  less  time,  and  are  easier  to  maintain.  Even  though  authoring  tools  are  the  most 
practical  option,  care  must  be  taken  to  select  a  tool  that  provides  the  applicable  features 
required  by  an  ATAPMS  such  as  the  capability  to  capture  and  retrieve  the  content  of  a 
user's  session. 
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VI.  CONCLUSION 


A.  CONCLUSIONS 

The  DoD  has  identified  the  education  and  training  of  the  acquisition  workforce  as 
a  strategy  to  help  make  the  acquisition  system  more  effective  and  efficient.  As  a  result, 
the  DOD  established  the  DAU  to  provide  the  required  education  and  training.  More 
recently,  EO  13111  and  the  DRI  have  presented  a  mandate  for  the  DoD  to  find  ways  to 
use  technology  to  further  this  goal.  In  concert  with  this  mandate,  the  DoD's  ADL 
initiative  will  provide  the  added  reach  and  the  decreased  cost  of  education  and  training 
through  the  creation  of  industry-wide  standards  for  educational  software. 

Even  though  the  DAU  is  responding  to  this  mandate  and  using  technology  to 
provide  interactive  distance  learning  courses,  there  is  one  type  of  course  not  yet  offered. 
There  are  currently  no  automated  courses  that  include  a  feature  desired  by  several 
instructors  at  the  NPS:  the  capability  to  allow  instructors  to  qualitatively  assess  a  student's 
work. 

B.  RECOMMENDATIONS 

To  help  satisfy  this  need,  this  thesis  developed  the  top-level  requirements  and 
illustrated  a  notional  Acquisition  Program  Management  course  for  an  ATAPMS  model 
tailored  to  the  needs  of  instructors  at  the  NPS.  Most  importantly,  ATAPMS  needs  to 
accommodate  many  interchangeable  modules  and  must  scale  to  the  user’s  needs. 
Additionally,  the  program  must  also  incorporate  the  Advanced  Distance  Learning 
precepts.  Table  6-1  provides  a  comprehensive  summary  of  the  recommendations  for  the 
structure,  strategies,  and  topics  for  the  application  of  an  ATAPMS  at  the  NPS. 
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Structure 

Content 

Strategies 

-Use  a  capstone  style  approach 

-Acquisition  Strategy  for  a  Major  Defense 

-Chronological  by  Phase 

to  the  content 

Acquisition  Program  (MDAP) 

-Concept  Centric 

-Accommodate  at  least  10 

-Acquisition  Strategy  for  a  Major 

-Systems  Engineering 

weekly  topic  modules 

Automation  Information  System  (MAIS) 

Centric 

-Be  capable  of  presenting  well 

-System  Engineering  Process 

-Logistics  Centric 

developed  thought- 

-Defining  Critical  Design  &  Capabilities 

-Hybrid  Approach 

provoking  scenarios  and 

-Analysis  &  Control 

questions 

-Risk  Management 

-Ensure  modular  architecture 

-Acquisition  Logistics 

for  scalability  and  a  plug  and 

-Contracting  &  Contract  Management 

play  capability 

-Budgeting 

-Be  IMS  compliant 

-Test  &  Evaluation 

-Ensure  Accessibility 

-Production  &  Fielding 

-Ensure  Durability 

-Work  Breakdown  Structure  &  Earned  Value 

-Ensure  Re-usability 

-Software  Management 

-Ensure  Cost  effectiveness 

-People  Factors 

Table  6-1.  Summary  Table. 


The  most  practical  option  for  building  the  ATAPMS  modules  is  through  the  use 
of  commercial  based  authoring  tools.  However,  care  must  be  taken  to  select  a  tool  that 
provides  the  applicable  features  required  by  an  ATAPMS  as  listed  in  the  "Structure" 
column  in  table  6-1.  Additionally,  any  tool  chosen  to  build  ATAPMS  modules  must  be 
capable  of  capturing  and  retrieving  the  content  of  a  user’s  session  to  enable  the  dynamic 
generation  of  graphics  and  an  instructors  qualitative  assessment  of  a  student's  work. 

C.  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

There  are  three  topics  that  require  further  research.  The  first  topic  is  the 
comparison  of  authoring  tools  to  determine  which  tool  best  matches  the  requirements  of 
the  instructors  at  the  NPS  as  well  as  the  DAU.  The  second  related,  but  distinct  subject,  is 
evaluating  the  need  for  a  system  to  provide  management  for  the  users  and  the  ATAPMS 
modules.  The  third  and  final  topic  involves  developing  a  working  prototype  for  a  course. 

While  there  are  several  authoring  tools  available,  a  careful  review  is  required  to 
find  an  appropriate  match  for  the  individual  consortium  school.  Because  it  is  possible  to 
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find  more  than  one  product  that  meet  the  requirements  outlined  in  this  thesis,  further 
research  is  needed  to  evaluate  the  candidate  tools'  strengths  and  weaknesses  against  each 
other  in  conjunction  with  the  needs  of  the  DAU  consortium  school. 

In  addition  to  an  authoring  tool,  further  research  should  also  include  the  selection 
of  a  management  tool.  Management  tools  are  used  for  the  "command  and  control"  of 
courses  and  students.  They  provide  a  method  of  centrally  controlling  the  various  learning 
activities,  including  course  delivery,  access,  collaboration,  and  performance  tracking. 

Most  authoring  tools  can  operate  independent  of  their  associated  management 
applications.  However,  the  majority  of  tools  encountered  during  the  research  of  this  thesis 
had  a  corresponding  management  application  that  was  more  expensive  than  its  associated 
authoring  tool.  For  example,  one  vendor  only  charged  $1,200.00  for  the  authoring 
software,  but  charged  a  minimum  of  $8,000  for  the  use  of  the  management  tool. 
Therefore,  the  utility  of  incurring  an  additional  expense  to  procure  a  management  tool 
requires  additional  analysis. 

Finally,  once  the  questions  surfaced  by  the  aforementioned  topics  have  been 
resolved,  the  next  step  is  to  develop  a  working  prototype  for  an  entire  course  offering. 
The  development  of  this  prototype  has  two  requirements.  The  first  task  is  to  gain 
familiarity  of  the  features  offered  by  the  selected  authoring  and  management  tools.  The 
second  task  is  to  work  closely  with  the  cognizant  academic  department  to  develop  a 
relevant  lesson  plan  that  can  be  supported  by  the  authoring  and  management  tools. 

Because  these  topics  require  careful  analysis  before  implementing  a  workable 
ATAPMS,  they  are  ideal  subjects  for  further  research. 
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APPENDIX  A.  SELECTING  AN  APPROPRIATE  AUTHORING  TOOL 

There  are  currently  over  90  different  authoring  tools  for  designing  and 
implementing  a  computer  based  training  course.  Selecting  the  most  appropriate  tool 
without  using  a  systematic  procedure  is  a  difficult  task  at  best.  One  methodical  approach 
to  this  task  involves  a  thorough  analysis  of  the  user’s  needs  and  then  locating  the  product 
that  best  matches  them. 

To  assist  a  user  assess  his  or  her  needs,  most  on  line  learning  sources  provide  a 
list  of  questions  to  help  illustrate  the  features  available  for  authoring  tools.  To  help  match 
the  overall  needs  of  the  user,  the  enclosed  tables  consolidate  a  comprehensive  list  of 
features  characteristics  available  in  authoring  tools. 

There  are  several  resources  available  through  the  World  Wide  Web  that  provide  a 
listing  of  tools  and  their  features.  For  example,  there  is  a  website  at 
[http://www.umr.edu/~rhall/research/webleaming/products.html]  that  lists  links  to  over 
20  websites  related  to  authoring  tools  and  products  and  an  additional  10  websites  related 
to  the  assessment  of  authoring  tools. 

Once  the  user  has  determined  his  or  her  overall  need  and  identified  the  required 
features  listed  on  the  following  tables,  the  next  step  is  to  find  the  authoring  tool  that  best 
matches  the  desired  features  listed.  One  site,  shown  at  Figure  A-l  and  located  at 
[http://www.ctt.bc.ca/landonline/option2.html],  allows  a  user  to  perform  an  online 
evaluation  of  up  to  33  authoring  tools  within  its  database.  Using  this  online  evaluation 
tool,  the  user  can  select  the  applicable  features  and  the  evaluation  tool  will  display  the 
application  name  of  the  authoring  tool(s)  that  most  closely  match  the  features  indicated. 
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The  other  option  for  this  program  is  to  enter  the  authoring  tool  name  and  the  evaluation 
tool  will  display  checkmarks  in  the  corresponding  feature  boxes. 

As  an  additional  resource,  an  online  glossary  of  terms  is  available  from  the  same 
provider  at  [http://www.ctt.bc.ca/landonline/glossary.html]. 


[Select  by  application  name  §H 

Learner  Tools 

Web  Browsinq: 

Support  Tools 

Course: 

view  IliliiifnS-  ¥ . ^  IllS^ll 

F  Accessibility 

H  Course  planning 

notes 

HI  Bookmarks 

n  Course  managing 

£y&|j 

c  1  . . 1 

F  Multimedia 

n  Course  customizing 

O  O 

fj  Security 

c  1  !| 

Asynchronous 

Lesson: 

i .  __ - - - - ^ 

Sharinq: 

FI  Instructional  designing 

c  c 

□  E-mail 

n  Presenting  information 

C  !  !i 

!"1  RRS  file  exchanae 

□  Testing 

C  C . 

F  Newsaroups 

Data: 

'  ;  - 

C  1  ij 

Synchronous  Sharinq: 

□  Markina  on-line 

c  c 

□  Chat 

fl  Managing  records 

F  Voice  Chat 

F  Analyzing  and  tracking 

O  1 _ 1 

fl  Whiteboard 

Resource: 

c  c 

n  Application  sharing 

F  Curriculum  Managina 

m ; 

c  i  . . . j 

F  Virtual  space 

F  Rnildinq  knowledge 

ill  I 

c  o 

r  Groun  browsina 

F  Team  Building 

iWC  ; 

C  1  1- 

n  Teleconferencing 

F  Ruildina  motivation 

W  ■■ 

C  1  .  - - ^ 

n  Videoconferencing 

Administration: 

c  c 

Student  tools: 

F  Installation 

M  ! 

o  1  — 1 

F  Self-assessing 

F  Authorization 

ipl  l 

c  6 

F  Progress  tracking 

F  Registering 

jB , 

c  1  :  _J 

D  Searching 

n  On-line  fees  handlinq 

|f  | 

o  c 

F  Motivation  buildina 

F  Server  security 

Ml 

r  1  * 

F  Studv  skill  building 

1  j  Resource  monitoring 

_ 

F  Remote  access 

c  o 

F  Crash  recovery 

m 

To  compare  2  applications 

_ i:  _i_  _ i — - <* 

(back  to  ton  of  forml 

Help  desk: 
r~  — * 

j 

Figure  A-l  From  [Ref.  46].  An  Online  Authoring  Tool  Comparison  Program. 
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After  selecting  one  or  more  candidate  tools,  the  next  step  is  to  contact  the  vendor 
and  verify  the  information  concerning  the  advertised  features  and  their  actual  cost 
structure.  Usually,  the  vendor  employs  a  sales  staff  to  field  the  rudimentary  questions. 
However,  it  is  helpful  to  obtain  a  contact  number  to  the  technical  staff  for  questions  that 
require  a  more  detailed  response. 

In  addition  to  gathering  cost  and  feature  data,  testing  an  online  demonstration  or 
obtaining  a  trial  version  of  the  candidate’s  software  is  the  next  important  step.  Most 
vendors  post  an  online  course  that  demonstrates  the  capabilities  of  its  authoring  tool. 
Navigating  through  the  “demo’s”  offered  through  the  various  candidates’  websites  is  an 
excellent  way  to  compare  the  each  vendor’s  product.  Ultimately,  obtaining  a  trial  version 
of  the  actual  authoring  tool  is  the  ideal  method  for  familiarizing  oneself  with  the  all  the 
tools  features  and  capabilities. 

Listed  below  is  a  comprehensive  set  of  questions  to  help  illustrate  the  features 
available  for  authoring  tools.  These  questions  are  derived  from  a  site 
[http://multimedia.marshall.edU/cit/webct/compare/comparison.html#develop]  that 
presents  a  comparison  of  online  course  delivery  products.  However,  because  the  specific 
products  and  their  versions  listed  on  the  site  are  subject  to  frequent  changes,  this 
appendix  presents  only  the  questions  listed  on  the  site  and  not  their  results. 
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ADMINISTRATIVE  FEATURES 

University  will  have  sole  ownership  of  custom  code  used  to  create  courses 

University  will  have  sole  ownership  of  course  content  _ 

Platform  provider  will  provide  technical  support  to  University  students 

Platform  provider  will  provide  technical  support  to  University  faculty  and  staff 

Platform  provider  will  host  courses  on  their  server 

Platform  provider  will  advertise  courses 

Platform  provider  can  provide  documentation  and  contacts  to  demonstrate  a  positive  track 
record  with  higher  education 

Pricing  structure  is  based  upon  number  of  students  within  the  course 

Platform  is  focused  on  locally  developed  courses  as  opposed  to  "canned"  courses 

Platform  has  large  startup  cost  with  minimal  continual  costs 

Table  A-l.  Administrative  Features. 

ADMINISTRATOR  TOOLS 

Server 

Client/Web  interface 

Authorization  tools 

Logout  feature 

Resource  monitoring 

Remote  access  tools 

Crash  recovery  tools 

Student  support  tools 

Instructor  support  tools 

Administrator  support  tools 

Built-in  file  management  tools 

Ability  to  export  raw  data 

Customization  of  text  messages 

Resume  session  function 

Security  access 

Variable  level  of  security 

Online  registration 

Registered  markers 

Batch  process  for  inputting  student  accounts 

Guest  account  creation 

Table  A-2.  Administrative  Tools. 
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DEVELOPMENTAL  FEATURES _ 

Content  format  will  allow  for  simple  transfer  to/ffom  another  vendor's  platform  _ 


Platform  uses  open  data  standard  so  that  it  can  communicate  with  existing  university 
database  applications  _ 


Content  can  be  authored  on  PCs  running  Windows  95/98/NT _ 

Content  can  be  authored  on  Macs  running  OS  7.5  or  greater _ 

Courses  can  be  taken  using  a  PC  running  Windows  95/98/NT _ 


Courses  can  be  taken  using  a  Macintosh  running  OS  7.5  or  greater _ 

Platform  provider  is  supportive  of  implementing  IMS  standard  within  product _ 


Platform  provider  is  supportive  of  implementing  AICC  standards  within  product 


Platform  utilizes  standard  HTML  for  content  creation _ 

Platform  is  structured  so  students  can  view  all  of  their  current  courses  when  they  log  on 


Platform's  server  software  will  run  on  DEC  Unix _ 

Platform's  server  software  will  run  on  Windows  NT  _ 


Multiple  choice  questions  can  be  created\scored  with  platform's  authoring  software 


True\False  questions  can  be  created\scored  with  platform's  authoring  software _ 

Matching  questions  can  be  created\scored  with  platform's  authoring  software 


Short  answer  questions  can  be  created\scored  with  platform's  authoring  software _ 

Essay  questions  can  be  created\scored  with  platform’s  authoring  software _ 


Platform  supports  question  database  for  management  of  test  questions  _ 


Platforms  supports  reporting  features  for  test  questions _ 

Platform  supports  Microsoft  Internet  Explorer  4.x  and  newer  browsers 


Platform  supports  testing  stage  for  courses  to  debugged  before  making  them  live  to 

students _ _ 

Platform  allows  author  to  view  course  as  student  without  logging  out 


Platform  has  built-in  threaded  discussion  list  capabilities _ 

Platform  has  built-in  chat  capabilities  _ 


Platform  can  be  integrated  with  Real  networks  video  and  audio  products 


Platform  can  be  integrated  with  Macromedia  Shockwave  products _ 

Vendor  provides  development  services _ 


Management  component  will  create  reports  for  tracking  student  progress _ 

Platform  has  a  feature  to  import  existing  test  questions  in  a  tab-delimited  format 


Table  A-3.  Developmental  Features. 


HARDWARE  REQUIREMENTS _ 

UNIX  server _ 

NT  4.0  server _ 

CGI-enabled  Web  server _ 

Java-enabled  Web  browser _ 

Mac  OS  _ 

Solaris  _ 

Linux  _ 

Table  A-4.  Hardware  Requirements 


99 


INSTRUCTIONAL  FEATURES 

Platform  choice  can  be  customized  to  incorporate  SPU  distinctives 

Faculty  to  student  asynchronous  communication  is  possible 

Faculty  to  student  synchronous  communication  is  possible 

Faculty  can  make  their  own  changes  to  content 

Training  is  provided  for  faculty 

Courses  can  have  consistent  interface 

Platform  supplies  access  to  library  resources  other  than  the  university's  present  holdings 

Online  help  is  available  to  help  student  use  library  resources 

Platform  includes  an  internal  e-mail  client 

Platform  has  e-mail  management  capabilities  for  students 

Platform  has  e-mail  management  capabilities  for  faculty 

Platform  supports  multiple  instructors  for  a  single  course 

Table  A-5.  Instructional  Features. 

INSTRUCTOR  TOOLS 

Course  planning 

Course  managing 

Fast  course  revising 

Course  monitoring 

Instructional  designing 

Presenting  information 

On-line  testing 

On-line  grading 

Managing  records 

No  HTML  knowledge  required 

Customization  of  student  curriculum 

Student  tracking 

Automated  grading 

Level  of  control  over  design 

Instructor  can  assign  specific  course  material  to  individual  or  group  of  students 

Multiple  choice  self  test  tutorial  questions  -  (automatic  marking) 

"Fill  in  the  blank"  self  test  tutorial  questions  -  (automatic  marking) 

Customized  feedback  to  tutorial  questions 

Redirect  path  of  tutorial  depending  on  question  answers 

Timed  quizzes  (graded  with  permanent  mark  retention) 

On  line  marking  and  grades  management  of  timed  quizzes 

Generate  random  set  of  questions 

Allows  developer  to  preview  course  as  a  student 

Table  A-6.  Instructor  Tools. 
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SOFTWARE  COSTS 

Start-up  costs _ 

On-going  costs _ 

Site  pricing _ 


Table  A-7.  Software  Costs. 


STUDENT  TOOLS 

Authentication 

Bookmark  management 

Multimedia  support 

Private  e-mail 

File  submissions 

Threaded  discussions 

Course  Chat  rooms 

Logged  chat 

Whiteboard 

Self-assessing 

Desktop  based  file  management  for  uploading  to  server 

Un-timed  quizzes 

One  question-at-a-time  function 

Bulletin  board/conferencing  tools 

Image  database 

Student  access  to  own  grades 

Access  to  course  grade  distribution 

Automated  glossary  tool 

Automated  index  tool 

Online  assistance 

Search  tool  for  course  content 

Student  presentations  area 

Allows  students  to  view  all  current  courses  in  which  they  are  registered  after  logging  in 

Table  A-8.  Student  Tools. 


TECHNICAL  SUPPORT _ 

External  e-mail _ 

Security  features _ 

Assignable  administrator  role _ 

Batch  add  instructors _ 

Batch  add  students _ _ 

Template  creations  tools _ 

Built-in  instructor  manual _ _ 

Built-in  student  manual _ 

Database _ 

Table  A-9.  Technical  Support. 
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